From nobody Sat Aug 12 23:29:56 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RNcMn2WMNz4m3xH for ; Sat, 12 Aug 2023 23:30:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-21.consmr.mail.gq1.yahoo.com (sonic317-21.consmr.mail.gq1.yahoo.com [98.137.66.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RNcMl13byz4dkD for ; Sat, 12 Aug 2023 23:30:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=AUlfcLIG; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691883009; bh=DnHtt4oyLOv+avQS+0iqUFclINWs/jQKvO1h2kiRJik=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=AUlfcLIGEHQJf6X6fLyv/WxPY+0eBnYtr4cWIOEwqPlCKmNNK5FM4whtLCWAejXwsbPNzEuTUzcCKRyUZKVBMoGPBWywYfDsTxMf/Z8uauWRYq8qMLK9k6tHZ1FzOTtFI/EYa0S9kHG4BFOFFunpwUQrJ8NC7wRPIDg1Qw8nq6EdLajpNo40iYZyr6DDeSFivhMyfBcdvDGKL7YgADmRjrfOzx9LDPSr/SgHG9CwfZc5ljKb2y+fbsofBkq++0gysZDCRvy0/hyrxhWdK6FEzLV+AK+XiYC0MkOxOY4vjwc3waQLIvD418IqUQrJ/zMGip5MEx7f7ZrWkrAOhMd8mA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691883009; bh=cLiom/oM90X+mZJz8LnPJC0dBvyLXJOb2T9X7QEUulb=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=CVDPyS4q0M1Ye4b1mfNU/EOsiEwH/mVRcXnaP+Ey3FzQf+2kteY+HMlupvAf/LUjQM+W98/HnRs9S06dZw1V9huQClctoHjJts2gFg5HV5zefKr6HE47w2T3givehX1OghUNke2LezxjgEe5CmZWL0KTEZTCsABBV8zsZftKRsSyNcfxcBYeQjkWHMRlVj8hBuAtffIfD5ZKhYV+Gze4gWaK1023qHId/3yI9+ouUmN4Q9zlt+xbA3Fo80hxMDduca8/+40bJ+JzARfjIjSWqBCuZ74DGUiHvErNEXgx8sCvb3z1jzO4pJMnEsGIiAGAsgSoFB57cGEt0dOBf5umaA== X-YMail-OSG: odhO268VM1nJ_KaT2a8mQXFj32MdMEKCGuLUHFTzhFUN1KPOOfHoQKBiKX630yi xxBHdk1yqOGl7ylnkPPVlBAvI_y9xViC9BlT51dg.FStW1S3gAmsbQ2.n6Vi7tgnVBQq7g2ZjSB3 7HklFu3rJMDYaUlrFka.OKGeRiJ3vAKza6EKv3FWv3A5979DeRqayfVx5zcy_emOiFPWfXmRLHvN oz_WAYmY9idyTtpX9tzA5.o4ylJMaQa2o92kFSNRdnoi.6zUDcNS8IOfqc0tCq.Igtut.lEwakDj iJ2wxsEzIVRdhulWJETaoY7fd5FqXnh8SVsYJgS9jUNZ2cBP6fqsPfdmD5pqXjnu0JhBlip.Ie5o 41mRZyE5CWpvuxrouqCxqpv7_EcV7vGTu4OjLE1ytoVYWo1aKKX7aCnmHyWUx2oLUyOd8IiD3zTM .lCVZpl0eTzRBgSKBmqVDcd0eG64pfRM7bbjeH2krZZPjtWfo095MR0ZOslSZlC..FCw11GkSCGD wFgG_zyE0cY_7wcGH40tTEKlj.NmRqf_glmahOyhxj.JgQNZqZcK5.7esTz63B8UhiEnm7LjDKmn QpSJhKTt1xUBU_V.Heu8UMUEteeLnR_g0sdasCikoGOdTfVFvFWszmL5FfRLPvmwCtK3GblR8LhR tuJ0r8ImsCWk3N5mfZif9tSs2KnWpMNkxgIiXENLofha.MuqZhtR3GKhqPVJZ3gVbE4IJUf5B2Rc KsKFK2c9Jx.jctXloW.ymiHGQi1fkv4.naJNNZ2.8eEjP8Ia2wU9AxEu.emlFK8Ugz7UP5w1XInx cGVbPv.kc4EgyJS7ar4WH6p2B__p5DGFXpUmojIcVojIRZ6fpLduttOCGiQmOH8LeubIIEH.Klin s_nx11a5P5TJrgMiXqdHhfjghTdeRXkG_kV.9y5ckk_n.Lx.s8aaycJc3L92FwcFJPwjUW2jckTj AWXtBptuv0_VQhc4U44BJkTZG0aDu1Ni9GARNCD28udVyATg1.q2X0yfwL9_MhQoeoG4fGMuPRXd n96rvdzoiGsNFqfH17a8lD2Oz.LJZNdLpGouf5ZklgefiPXKVww.avSL497P5uPxvGAA2H3_cUd0 nUm.w17f.Iiymw40H5OtrUDq9A7cLRgxFjeE7p7k7ZBYjJ7S9NINqJT7VG0FDlMQbt6XNgi2oJkJ L4aoT.wlDqvErPjnqj5gL7hdeqs1KfGXkCzcnu.86avOfNvVDe6cX9gTK2aNuuCwBJ32bAlfTGLu Olq73lCne_ReSDgnOPPC3vKy1FBba1NkPkltltX8lUd4YwcvPrnsLHVe0O_4k4rx3Zd5IfsdkTKw ignC4xTmz40GCPG2kKv29AULqNEJj1W.Qqq9mEN7wvu14Xt8QYw69TMPRH42OwBwtku1fMVXUmF. R4cuFNaPEYxC1_.lIbwFN2vb4eViNYChxnQREmNt.lTsvC14Uj9oG53B8.7A0zYu0HXuAzz4rsDi N.kUwGEjJnyLOJMKe6XYP8l_WEVlXL8Up1PpNsP.G5D9hRP3Vp9xTZsuDUzmXKFFSKUwH59_c_zq DPhnl02cojO1ULFFEPuhcRoD6KCc.j2pFj1GCADUDImz9xxsRE_oG4evaH0UeCjhKDa42aLGLQhF VL8A_Gslv8m_Fva5jC9Ra6nfcQ3FenKNvOXkj3DBe5wqTgrhRGCKwOhi4jeYJcHEd2symZraYSob pWeV1AIm26yLC3DvzD6pLq05NAKzw38oy8TV1JeKOGi85HQw4y0.JWzOyIc23FzH0Nee7PBMMfg2 BIOgJ6d2s0RI2ky6iEIqQPHe736uLReu2J6dPO0_VDkeBxLJmWGW8UPy.DpFBxmEeIXpr9eCxh2H ekHWCyeFuZEU60BcvJjOR4zLeoQT9KH.ihB3MTF_NCPbOnbB6yNVXPII5M27nCN7vJGbVVlvQ4kZ kNx3ZJDE7jkv40jfAm5txF4uHNbN8YD2ckm..gfM0MCaAte0.FAx9HLSFY.aUZfnhypGyWPhUoK7 A1vzTO4llQIGiUf.ix0jj.OoOECBfmm8Aqe3u0tixNsMB7W57uSOTd4Igod1DvZah0AQPqEzATwm lUOUcDyi1AxsBajSnuqDXdqQsdEWLCzH_bXoTHFXvWA6P5vq0_KdPR3KeJNtXGzqmvtS24iKkah3 cyAXlRX8NfjWiODhrhQBQaCrhiubgTwkuTg3etifvA0xgv1Z.4DOuforQo97ktwRybFAAPfIT_A8 KltCvsbiQdGzqPRPRXYXW97YRa13MGKSiZVWe0rZr1EKgGXAvN5uHMZ4aODKkwEjHXc.U5vJCyvH gT50- X-Sonic-MF: X-Sonic-ID: e9e73837-9ab0-449d-8808-ab7a275d80d7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Sat, 12 Aug 2023 23:30:09 +0000 Received: by hermes--production-gq1-6b7c87dcf5-6x8bf (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID d9fa567403029c6c445f4f95d8243d7f; Sat, 12 Aug 2023 23:30:07 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 Message-Id: <84743BEB-8395-45B6-8B9F-4AAB082076A1@yahoo.com> Date: Sat, 12 Aug 2023 16:29:56 -0700 To: Mike Karels , Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: <84743BEB-8395-45B6-8B9F-4AAB082076A1.ref@yahoo.com> X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.989]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.147:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.147:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RNcMl13byz4dkD Mike Karels wrote on Date: Sat, 12 Aug 2023 22:17:39 UTC : On 12 Aug 2023, at 15:32, Juraj Lutter wrote: > > Hi, > > > > recent 14.0-ALPHA1 sources is giving an error: > > > > Building = /usr/obj/usr/src/arm64.aarch64/obj-lib32/lib/libssp_nonshared/libssp_nonsh= ared.o > > Building = /usr/obj/usr/src/arm64.aarch64/obj-lib32/lib/libgcc_eh/int_util.o > > error: unable to create target: 'No available targets are compatible = with triple "armv7-unknown-freebsd14.0-gnueabihf"' > > 1 error generated. > > error: unable to create target: 'No available targets are compatible = with triple "armv7-unknown-freebsd14.0-gnueabihf"' > > *** [libssp_nonshared.o] Error code 1 > > > > Commandline: > > make -j`sysctl -n hw.ncpu` WITH_META_MODE=3D1 TARGET=3Darm64 = TARGET_ARCH=3Daarch64 buildworld buildkernel > > > > The build runs in clean objdir, src git hash = 220427da0e9b2c1d8e964120becc17eb7524e46f > > > > Host runs 14.0-CURRENT 28d2e3b5dedf > > > > Am I missing something obvious? > > Thanks. >=20 > Did the buildworld start out by building a cross-compiler? >=20 > Have you tried without meta mode? With a clean objdir, I don't see how > it would matter, but I'm not sure I've tried it. >=20 > The ALPHA1 builds seem to have worked, but I think they run on arm64. >=20 buildworld buildkernel is always done on amd64 for all the official builds as I understand. No use of quemu is required for this. https://ci.freebsd.org/ allows looking at the build logs and such for the ci build activity, not for the release/snapshot builds. It is ports being turned into packages that has the issue of wanting to avoid qemu use and so that only armv6 builds ports->packages via amd64. https://pkg-status.freebsd.org/?all=3D1&type=3Dpackage allows looking at the logs and such for this. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 07:32:39 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RNq4t0Bf2z4q8QP for ; Sun, 13 Aug 2023 07:33:02 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-8.consmr.mail.gq1.yahoo.com (sonic315-8.consmr.mail.gq1.yahoo.com [98.137.65.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RNq4r3lsMz4Lfs for ; Sun, 13 Aug 2023 07:33:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=bv+nXCEu; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691911977; bh=JMNukRgTG1hWRK/8XiUL7qZDQVdbdvtP4re9i6kVhW0=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=bv+nXCEuj7zPzeRbbj1w/QdOabHWhp1yGuzMTzDjifbLjlYCds6QKsDxMQNz9wgT0Gt5LNbg2+OYlZNsdEBjfFigmoe8b6eNN26O1GA2rz78iltaRV2QvbcT1iKMdKNkoYsiT1+VVLga4brs40EA8dR7YRv7xMfel+dk5FUBrPp4bXO7xhbWsvfWLl1BmWMvFdgzDTVW1uZVofLyAM+0F1l6wCuWfE19zuVG3NWYQj2lnCK+Bn8yB2Gn5AzTmxEDocD1cg5K9U/+hzc5iYeAzD8RMsxDYZF/SshksmqRiMRSausGYTOw4dGsqG3vKqfPo6143CUCJFdruKAjfS0OCg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691911977; bh=sJoRgiBH3rrwAKzgBKPwGE3UGSydMGOlIGPd1qXxT/w=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=GTHlFp7CQnBXRPZPE7grvSehbq1Z98oDPy3cPMg32FPvgwF0APzqo3Ro8W2o+QMYgNJA9jfHciL0sY+UZ121W/8crG9FjjAq1o7FGq4F9b/vlnwf6fN5Rbam3B9AU9t3oEX5/CYLCo9fw5TpvlRPw7brKYa2pkwmxSRNCwakoPmTmkan95caIP437AFJONod3sZGQXRHhfzgqJsYobKo/7wC/E9XFnNaYUp5sQXCfBVFlt2fkCF2LgEVBu+0gahvZsqeBgB0dj2SneJ6fcKg2kyrnYn/Bc6nZo78NIzOk4WKhbofE/Nv0u1brQsr70+BhjwUM/1EVj+HnrLOqJycOA== X-YMail-OSG: gG8j4ogVM1ny_IwijaFdElYAWKmctMYMtztlEZYhMeEq.8E7791iYT0S91gs524 L3x88Hk.SG_o01x.PvzX6vL1sWzCH1RK6LpcMv1rU1l7CddyqMmZ3ujvCNyWV.HgfIhg5WeKNMOR Zif7FJUV9UdpbwdUqVTWW2Ibpop8j.n8NTsNBkFe6WALZhoEpIgdPkPIwzZ1Qp3hFxF7nk9Sdung oPWug914bMEzSEjvWhmhd2_GWgRQOz12tY8hHbUNuPJKfHiqCgEPEmyjQT5gpE4YYjIOeBnaMtkU AY7.YIYNkuMSRU2cOiZl1QFm2QS2IkSJaSEKt0cFNJHW9ZdUTy5ez0IN0hknNJ70KZ9YnwbSkYDG FfbwsZ1GO_H69LcrmGS3C9jr8CEdD9S7aYu3ecVBfzafoBD3qh2N8D3rdKXQKKm7fxMoYZuKJPT_ QpXrrYfiEE8XHuutMLvK8jh5kYDcJQ3pZujbLRWbYF7sLqMuJOIwqcespW7qCDgM47ks83dh0xil 1pXXl6RE5wdNarC_PwIurujEyAo18cUWMblN43OUZpjewYtqpLsT0_lcAFBt6.3Jqex0Yr3IBtsL E2taxXD0tg0_.XtOWxxS12Is1SN9EGePNv7Umbq_0Qi_o34xV2Nz0iZaweZ0foaDh.Gx20ubqRUz Ej.WFFfThANu9WaushlGCuk2EWpBfZAqW.aeF2Fo342GhWy4x2jWECgrXdx4j.f_EddQbKhgIdiA ygLicAyvSFwPbmGyeuICl7tOeZ09xqIxBDcbhyJ_uoMGvfoiPyknUey70kr0YK2ZY629cnSUe2vK ECMZzrbmdjdK0JYFw1asauiS1EmYvFGmwiSfVu3Cr8QtEF8IdvfpvkwRWlfGK5aUaowPOGoL.4r9 zOw9O_JCfMYYaIze2rNb2YbW2jYM48TLNMp5ASi72gdtzfGhQUjfmYwQ3Sg_gweYm1nZ1RqUfsRT wgBc9Y3F2RomEATcdg31c_jQKDZgRB9bHA2Q563OWBUeg6.5wYHQ2DBCYuD69Dzh9WgXN4Mtmm4N rSj5BdijaiH3PsdQG91PEX2LI8mUouX_RZgk.DqiCsNr1kJpFmmP8uI3wnGoNXuK2in7JaJXSCeS KBaySuD0uTsyxsF5oXW4QVjfn.WupSxu2onfK1waV6HxDWwYnX7764JX24WY5Qlx6puuXv1WZ1VT AHbXA4jGqOCVCK62XpBc05MJtzWxYPDHNbeOVUkR6oUtJSU4n7msQKAewXP74tD2q7Bw_JCvSrwy EObLF3aJcMV0tp94DWykbMN0OlBDfbC4TBtfSDOdySW92Y6XdcDJCbozlk2sa31Y4Ut8XJacY5yI 2jSX8osVieJ_xaG4ZqmLf4BFIE15gcosKdyEdCy.peEfbURqkIPOH__ieXFNCIdqKNRE3drE_uns Um1BQ_VdxSm01fasNzQcTWQ9q2JWGDeGEMlIjOvMbNfC.A9ftC0C24VI3VUCd7taDJPuY91DprlP KzAbnmKqJMjiT5ZhIQXxj5e6RgF4FzwtDlEMM2GvlkwPtI2Sr8QiDfMy65L4KRHPLoXVGBD2w5a5 zxDaSnE.yBNe2i8WF4Uc1lnbxgr095SIBge0RsENxuOwYIC12.cZP6s60SLVxpR2eUrwSIKogRca 33prRrlehNG1BR.7Hsk5W_mdssz3yS_Gtv_pfbFoB6Am2GHQlbui928nG2DaoJu5frsAqzVmo8uF 3l7U18sJRhjk97_cquFStecnEtDQ9UKKQ.vcMoX_SALyN8I9pN5k6yEAh.JJh3S27kNxtx.TQEm9 GeA7QAi2yBHndmZpVScVodAwP_I1vg4b8nkOMPJbMyblnWVY.9MSlC2_de4DmynaePk1UideqpS_ OqG9s8b.U_YyI3koC7muObDaw8QvgxH_UIiFMzFqcekQQxaaYeKvKXHeK6H5zrgxefF.1IIiOIpx 3hypMK5hoqdJANCWGaJYFG7SLOHNslQutUOmoiG5POLMWTEU5pLFCh2loTk_w_EwuaI3ZI4ZXmNE yKrGNUVua9_.58IZpMfRnMi5dsTI.zj8W9lQHdxaH5MH4QtDPrx5.nxnmZvcKEu5wY1.1dHPophb VA2SPy25L3sST0.Ql1KlFSzZLB8Q7Nvft2ToEVatu9P5DreG6yCpEZ11YyYPGpZjclVsLKibMJWu o6MDA..F1M_Ca3hx1JxkGPgdCaF8p5mm19yT2AZX6EVbSvZF0gVt7xQFPPzscDqXLiOtF14mMYzw gK5iK.jYqoCS9SYPxOwPGx_X0YMFX7ouWkPselTNRbyYQN6.KPIGIGDtQXtDg X-Sonic-MF: X-Sonic-ID: b1648a59-d5eb-4a7a-8ec5-c29b5bcb1a21 Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 07:32:57 +0000 Received: by hermes--production-bf1-865889d799-k7hdq (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 36f248d8d29a1bc039e3a9e08821a0c7; Sun, 13 Aug 2023 07:32:51 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: /usr/sbin/etcupdate -D/usr/obj/DESTDIRs/main-CA7-chroot -s /usr/main-src -M TARGET_ARCH=armv7 ends up building dtc ! Message-Id: <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2@yahoo.com> Date: Sun, 13 Aug 2023 00:32:39 -0700 To: Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2.ref@yahoo.com> X-Spamd-Result: default: False [-3.48 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.984]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.32:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.32:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_ENDS_EXCLAIM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RNq4r3lsMz4Lfs I'd noticed that the etcupdate part of my build procedure has been = taking much time. So I looked with a ps -axldww while it was going on. An example was: `-- /bin/sh /usr/sbin/etcupdate -D/usr/obj/DESTDIRs/main-CA7-chroot -s = /usr/main-src -M TARGET_ARCH=3Darmv7 `-- make TARGET_ARCH=3Darmv7 -DNO_FILEMON buildetc `-- make -m /usr/main-src/share/mk -f Makefile.inc1 TARGET=3Darm = TARGET_ARCH=3Darmv7 buildetc `-- make -f Makefile.inc1 _bootstrap-tools MK_CROSS_COMPILER=3Dno = MK_TOOLCHAIN=3Dno . . . `-- /bin/sh -e -c echo "=3D=3D=3D> usr.bin/dtc = (obj,all,install)"; cd /usr/main-src/usr.bin/dtc;. . . install `-- make DIRPRFX=3Dusr.bin/dtc/ all `-- c++ -O2 -pipe -fno-common -DNDEBUG . . . -c = /usr/main-src/usr.bin/dtc/fdt.cc -o fdt.o More may build. This is just what was present when I happened to look. Seems an odd thing to be happening for an etcupdate . I'll note that aarch64 targeting aarch64 (self hosted build) also spends notable time and is likely doing similarly. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 08:06:43 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RNqqt2dFbz4qBjf for ; Sun, 13 Aug 2023 08:06:50 +0000 (UTC) (envelope-from freebsd@igalic.co) Received: from mail-40136.proton.ch (mail-40136.proton.ch [185.70.40.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RNqqs730xz4PpK for ; Sun, 13 Aug 2023 08:06:49 +0000 (UTC) (envelope-from freebsd@igalic.co) Authentication-Results: mx1.freebsd.org; none Date: Sun, 13 Aug 2023 08:06:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=igalic.co; s=protonmail2; t=1691914007; x=1692173207; bh=Uvt9liWWi+AsT8uevmggiQB9nYs4KsDbnW26ZU06Yfw=; h=Date:To:From:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=tZv6eyeJoKp8cfE//PtR+fzAGgZKY/2VGK5FavOgNywFi9e0LGBNQItgVBDc24mxq QDP3PmFI5QjgvbUZoWKfvYImTH5wi6n0LLtuXJgj6lcLIcReec3U3hf1vL8UCaf5Sy vJrjM3M0WZeoqvbeWe82S4sw7/aEp9KXoYaLGN1eGIgHqBuTdK2hqnFu5huH0GUsHM pr4N1M/u5sD7ToWMrFABuNJ/VkW+Z9rWvrqNBpfDPtxUVujHAF3PiCQJKOd/tGxDeV QllQGZrxGxRu+GnyjNUlS2bZt3Ps0jfGQ4uXBQON91CTsyJKfDxSbuZU8OOJhFXai4 DPxa3TxsEJXCw== To: marklmi@yahoo.com, freebsd-current@freebsd.org From: =?utf-8?Q?Mina_Gali=C4=87?= Subject: Re: /usr/sbin/etcupdate -D/usr/obj/DESTDIRs/main-CA7-chroot -s /usr/main-src -M TARGET_ARCH=armv7 ends up building dtc ! Message-ID: In-Reply-To: <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2@yahoo.com> References: <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2.ref@yahoo.com> <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2@yahoo.com> Feedback-ID: 66573723:user:proton List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_awFmMyykJLlHgO9GS8m6M538SJ6N2FwLAdt5IAfHaU" X-Rspamd-Queue-Id: 4RNqqs730xz4PpK X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] This is a multi-part message in MIME format. --b1_awFmMyykJLlHgO9GS8m6M538SJ6N2FwLAdt5IAfHaU Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 ZnJvbSBJUkMsIGkgaGF2ZSBsZWFybmVkIHRoaXMgaXMgd29ya2luZyBhcyBkZXNpZ25lZDoKZXRj dXBkYXRlIHdpdGhvdXQgLUIgZG9lcyBhIGJ1aWxkLgoKQWxzbywganJ0YzI3IHJlY2VudGx5IG1h ZGUgaXQgc28gYm9vdHN0cmFwcGluZywgYmVjYXVzZSBvdGhlcndpc2UgQlVJTERfV0lUSF9TVFJJ Q1RfVE1QUEFUSCBkb2Vzbid0IHdvcms6IGh0dHBzOi8vZ2l0aHViLmNvbS9mcmVlYnNkL2ZyZWVi c2Qtc3JjL2NvbW1pdC9kODFkYTRjOTgzMjhkNGVlM2ZlNGMwYTg1ZjY4NzRhM2M2OWExYWZkCgpL aW5kIHJlZ2FyZHMsCk1pbmEgR2FsacSHCgotLS0tLS0tLSBPcmlnaW5hbCBNZXNzYWdlIC0tLS0t LS0tCk9uIDEzIEF1ZyAyMDIzLCAwODozMiwgTWFyayBNaWxsYXJkIHdyb3RlOgoKPiBJJ2Qgbm90 aWNlZCB0aGF0IHRoZSBldGN1cGRhdGUgcGFydCBvZiBteSBidWlsZCBwcm9jZWR1cmUgaGFzIGJl ZW4gdGFraW5nIG11Y2ggdGltZS4gU28gSSBsb29rZWQgd2l0aCBhIHBzIC1heGxkd3cgd2hpbGUg aXQgd2FzIGdvaW5nIG9uLiBBbiBleGFtcGxlIHdhczogYC0tIC9iaW4vc2ggL3Vzci9zYmluL2V0 Y3VwZGF0ZSAtRC91c3Ivb2JqL0RFU1RESVJzL21haW4tQ0E3LWNocm9vdCAtcyAvdXNyL21haW4t c3JjIC1NIFRBUkdFVF9BUkNIPWFybXY3IGAtLSBtYWtlIFRBUkdFVF9BUkNIPWFybXY3IC1ETk9f RklMRU1PTiBidWlsZGV0YyBgLS0gbWFrZSAtbSAvdXNyL21haW4tc3JjL3NoYXJlL21rIC1mIE1h a2VmaWxlLmluYzEgVEFSR0VUPWFybSBUQVJHRVRfQVJDSD1hcm12NyBidWlsZGV0YyBgLS0gbWFr ZSAtZiBNYWtlZmlsZS5pbmMxIF9ib290c3RyYXAtdG9vbHMgTUtfQ1JPU1NfQ09NUElMRVI9bm8g TUtfVE9PTENIQUlOPW5vIC4gLiAuIGAtLSAvYmluL3NoIC1lIC1jIGVjaG8gIj09PT4gdXNyLmJp bi9kdGMgKG9iaixhbGwsaW5zdGFsbCkiOyBjZCAvdXNyL21haW4tc3JjL3Vzci5iaW4vZHRjOy4g LiAuIGluc3RhbGwgYC0tIG1ha2UgRElSUFJGWD11c3IuYmluL2R0Yy8gYWxsIGAtLSBjKysgLU8y IC1waXBlIC1mbm8tY29tbW9uIC1ETkRFQlVHIC4gLiAuIC1jIC91c3IvbWFpbi1zcmMvdXNyLmJp bi9kdGMvZmR0LmNjIC1vIGZkdC5vIE1vcmUgbWF5IGJ1aWxkLiBUaGlzIGlzIGp1c3Qgd2hhdCB3 YXMgcHJlc2VudCB3aGVuIEkgaGFwcGVuZWQgdG8gbG9vay4gU2VlbXMgYW4gb2RkIHRoaW5nIHRv IGJlIGhhcHBlbmluZyBmb3IgYW4gZXRjdXBkYXRlIC4gSSdsbCBub3RlIHRoYXQgYWFyY2g2NCB0 YXJnZXRpbmcgYWFyY2g2NCAoc2VsZiBob3N0ZWQgYnVpbGQpIGFsc28gc3BlbmRzIG5vdGFibGUg dGltZSBhbmQgaXMgbGlrZWx5IGRvaW5nIHNpbWlsYXJseS4gPT09IE1hcmsgTWlsbGFyZCBtYXJr bG1pIGF0IHlhaG9vLmNvbQ== --b1_awFmMyykJLlHgO9GS8m6M538SJ6N2FwLAdt5IAfHaU Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 ZnJvbSBJUkMsIGkgaGF2ZSBsZWFybmVkIHRoaXMgaXMgd29ya2luZyBhcyBkZXNpZ25lZDo8YnI+ ZXRjdXBkYXRlIHdpdGhvdXQgLUIgZG9lcyBhIGJ1aWxkLjxicj48YnI+QWxzbywganJ0YzI3IHJl Y2VudGx5IG1hZGUgaXQgc28gYm9vdHN0cmFwcGluZywgYmVjYXVzZSBvdGhlcndpc2UgQlVJTERf V0lUSF9TVFJJQ1RfVE1QUEFUSCBkb2VzbiYjMzk7dCB3b3JrOiA8YSBocmVmPSJodHRwczovL2dp dGh1Yi5jb20vZnJlZWJzZC9mcmVlYnNkLXNyYy9jb21taXQvZDgxZGE0Yzk4MzI4ZDRlZTNmZTRj MGE4NWY2ODc0YTNjNjlhMWFmZCI+aHR0cHM6Ly9naXRodWIuY29tL2ZyZWVic2QvZnJlZWJzZC1z cmMvY29tbWl0L2Q4MWRhNGM5ODMyOGQ0ZWUzZmU0YzBhODVmNjg3NGEzYzY5YTFhZmQ8L2E+PGJy Pjxicj48YnI+S2luZCByZWdhcmRzLCA8YnI+TWluYSBHYWxpxIc8YnI+PGJyPjxicj48YnI+PGJy Pi0tLS0tLS0tIE9yaWdpbmFsIE1lc3NhZ2UgLS0tLS0tLS08YnI+T24gMTMgQXVnIDIwMjMsIDA4 OjMyLCBNYXJrIE1pbGxhcmQgPCBtYXJrbG1pQHlhaG9vLmNvbT4gd3JvdGU6PGJsb2NrcXVvdGUg Y2xhc3M9InByb3Rvbm1haWxfcXVvdGUiPjxicj5JJ2Qgbm90aWNlZCB0aGF0IHRoZSBldGN1cGRh dGUgcGFydCBvZiBteSBidWlsZCBwcm9jZWR1cmUgaGFzIGJlZW4gdGFraW5nDQptdWNoIHRpbWUu IFNvIEkgbG9va2VkIHdpdGggYSBwcyAtYXhsZHd3IHdoaWxlIGl0IHdhcyBnb2luZyBvbi4gQW4N CmV4YW1wbGUgd2FzOg0KDQpgLS0gL2Jpbi9zaCAvdXNyL3NiaW4vZXRjdXBkYXRlIC1EL3Vzci9v YmovREVTVERJUnMvbWFpbi1DQTctY2hyb290IC1zIC91c3IvbWFpbi1zcmMgLU0gVEFSR0VUX0FS Q0g9YXJtdjcNCiAgYC0tIG1ha2UgVEFSR0VUX0FSQ0g9YXJtdjcgLUROT19GSUxFTU9OIGJ1aWxk ZXRjDQogICAgYC0tIG1ha2UgLW0gL3Vzci9tYWluLXNyYy9zaGFyZS9tayAtZiBNYWtlZmlsZS5p bmMxIFRBUkdFVD1hcm0gVEFSR0VUX0FSQ0g9YXJtdjcgYnVpbGRldGMNCiAgICAgIGAtLSBtYWtl IC1mIE1ha2VmaWxlLmluYzEgX2Jvb3RzdHJhcC10b29scyBNS19DUk9TU19DT01QSUxFUj1ubyBN S19UT09MQ0hBSU49bm8NCiAgICAgICAgLiAuIC4NCiAgICAgICAgICAgICBgLS0gL2Jpbi9zaCAt ZSAtYyBlY2hvICI9PT0+IHVzci5iaW4vZHRjIChvYmosYWxsLGluc3RhbGwpIjsgIGNkIC91c3Iv bWFpbi1zcmMvdXNyLmJpbi9kdGM7LiAuIC4gaW5zdGFsbA0KICAgICAgICAgICAgICAgYC0tIG1h a2UgRElSUFJGWD11c3IuYmluL2R0Yy8gYWxsDQogICAgICAgICAgICAgICAgIGAtLSBjKysgLU8y IC1waXBlIC1mbm8tY29tbW9uIC1ETkRFQlVHIC4gLiAuIC1jIC91c3IvbWFpbi1zcmMvdXNyLmJp bi9kdGMvZmR0LmNjIC1vIGZkdC5vDQoNCk1vcmUgbWF5IGJ1aWxkLiBUaGlzIGlzIGp1c3Qgd2hh dCB3YXMgcHJlc2VudA0Kd2hlbiBJIGhhcHBlbmVkIHRvIGxvb2suDQoNClNlZW1zIGFuIG9kZCB0 aGluZyB0byBiZSBoYXBwZW5pbmcgZm9yIGFuIGV0Y3VwZGF0ZSAuDQoNCkknbGwgbm90ZSB0aGF0 IGFhcmNoNjQgdGFyZ2V0aW5nIGFhcmNoNjQgKHNlbGYgaG9zdGVkDQpidWlsZCkgYWxzbyBzcGVu ZHMgbm90YWJsZSB0aW1lIGFuZCBpcyBsaWtlbHkgZG9pbmcNCnNpbWlsYXJseS4NCg0KPT09DQpN YXJrIE1pbGxhcmQNCm1hcmtsbWkgYXQgeWFob28uY29tDQoNCg0KPC9kaXY+ --b1_awFmMyykJLlHgO9GS8m6M538SJ6N2FwLAdt5IAfHaU-- From nobody Sun Aug 13 09:23:54 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RNsY76gqKz4qHRQ for ; Sun, 13 Aug 2023 09:24:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RNsY742kqz4YFS for ; Sun, 13 Aug 2023 09:24:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691918649; bh=6jcL/aaHl6JS2yOXJp9oeSKe+R3vK2KtKBp5NUUunis=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=SNp5m9gfYKYQB1nts+93hb0+jkcaBwndBOI5Aqakv4IazkXCQy7VnLeGfYwlDlS64vRCBgBlCBmAaJlQKxi1OYiKijAi4koREDY0EajxYsagtVoDvAgnDIJnEdpXsq565oAzrNrNIf4GR9KFqPxawQ9FFOQflc/hqRA3Zec/qEBE6vF7h090e4Iv9x5qwDpXa3Pi6Yt6TGGk3psmQALKL7JAyuxXB1NyjNHU483GUVl6Bvi+dVeVW6a1Ibb0oP8akst4LQd1vSQsyXSfkDgLloMUhcrxtnV8Qdo7ZJB25nvmJA7zyD83fASwSw6oz2wgtbemzEweMiJ+L1TLuOlR1g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691918649; bh=5kQR7qdyrYCZEJ6vU6G3HxyONAdHrTaPh5ZTPwKRdXP=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ocROWxfT+7J6SlRgewftAmeVKMBJfZdkU8I/rmi3c71tMXs53LpC0/sD6KieGWcD3krdDhXBVaMHJLB7KBGaHyxlhWfamtMLufneruNWRwhOhoRFRNy8w/cGJ0TH4nDvDnm9ITQtSojvpIWJTov4gX4RdfFKPHbIId2ONPuB8V2BJG5AztuZSls0kQqKbQvS9HoPgQGQ+qceFLCF9f9nwC34qr8/fvhikNPkXUhKgrLqtO7Td8HdOs+4/jYqRpVFt9JXwZQj/D0zvyYH1RDtCvrH3VXGjvbM3yP3qSxR5VGI5JL53wNiaK8BFIxMQbUC/CgVxW/s9pEVksjzAEkPmg== X-YMail-OSG: tbNM8HsVM1lk2RD3MkPRRua46loocmxpaaZ39f5bDX2bkoQPcloY0z.Nrv_GQjZ 3rer4R6kdpT4hvR8PA3a.6qlTdlnSa_0QTVlTQiz9BBZ.8LZ_Yw3pOicdwq2XTjIsUkoK.tIqFBU D1Xi2B_26TwKeuVrgGUhyo23Fg0bvgVUXNGeZI3PH2TWy_KEO4dRkh6jcnUwoBmwseVkHRp2a2kh UmQbth2GKEC0TXoTOoien169.bmefDkZFbe7KHplb6NuSD9ZWMShpfXJM_oaFDHG9OgfY.gZwsCm Crc7GMf1XXk0aitYcKPTKGz3wUXgaSUK1SGc.FtqXatgGiULrDgL6IvXXmBWvkh8yWiURNygxaKx S67K.ax3HzcruqNYiGfkOYLt_DdJEJ3umTP_jOGGIUZn_SOfUfcq4sbBqeIv1GTx0boE0HzvuuDl DgMYWNs5PNUVjicrQ6lt4GSWtvC52j1mvCaAUR1m_Fv9ejl7JIyEpXDn9oXgKRYbF5XwMQMJPAo8 yFc4HZCeYV03Dyseml6niFh_EUoXBx5ttyX4BpJhNtr4YRlKFtFL57BQ4Y7yEKueg1poYZtVtXZw hKglBiNk69zmkX.GSMn7SY.xBLYiV4yzLHFALyGj5gytFGpBUTgTIc6jcGTL6iYYeDXTGCrzr8jL y_fxMb_be9Iff.yx_skE_d6W9cgGDIztps7BgwDaM1czmSawax3oRTey0yDVKOdJSs9g1ao5qdfZ v6tNv1wqs0B_0Bm3JltZLj0Zfh_PrDYHIQpgwKvgPQj4gaIdUON7MX98S8GrZAe5r9LNaXPnBdxh BkXPrCyhA2Dy4B6IDAS5DiFNm2aeZWx6kOHLCMcEl0bqujXK7okoFznRqBEfvdRd_DwlDsNsOP.u ktfvhkAwoGjHYovvE5Xj2pEWruKKK.w0BnfQjxxNJ1evYks6vZx8vU2tZUPmV4jx8nRvuXBXHylw 4HGow4hB4VUBJkyCYLNQ_I18b1QHPALsmnZ.6QPQ6fcpxWHhCV0MPVzMZhvfTsQ7OBiqN7UjZKVy 4Urjyiq2jJWAjXo067Ni48g0hYw8YqnNnW2REGLveyos1xpi2YGPxQB0dssdUxsYc5E5IgH_zmHi .jqe2515yOkKJsKqvHPUctBP3LR2Jl9nSuXctCw0kXt8NXSA3a7PTwg067cxeR584FYWDvDcTKTn WkGHfcagsndZ9iPRxnyE.4PvzkiVCE12rBXa8D2aPSH0R3fVMxbhSZiVP035wLc4.clPO1UvEi1H aUcRnGVgjalQo5K2Ih95omssKMkJns8pyIci.ErniFWgsFwx6QxcL_diQa6YFrerg0CXe69kv5dp 7u_NmiFLbB5weNd8vFMJmCVB7Utabd_i7SMF8RywZfw_iIcfh.Ha1ySvAuaKIXzJtUuOIXlPMm.b rrB9T.aiIdnofGmwzsn7CMvc0MZom_V0ylBpDQ2tWVBOUIfyuu5OmyZAsk03S0dgPCF0X7R2olJ6 z1DYrbAwPYk2n4X9F7RMzZhmQFl1b.G.6R0Qx1s4U42ypEQ8Khf7Uut5qtT5stFkkJmVMghYVKbG RLi_DplL4Vw3lO3y_VKj7Tv0vJpvL5eFs0XNSHeLBmSgeaTt3tK2Twa3xdhtdqfdN4muLOjfGYsZ 8bNCvNAkfSPmnYPpHt9W1Lxp6r1YXnjdbgYE7yE0TsP3kgCcQJRsuC2ri.gYg2R4BX8l.x2jmUnE fNQIAYeajditBgYzlATuga5JNM17QCwKJ6Gh0.tz9LZkgbmrHNG4YnEG1d.gaMEHya76T1_ZYCI6 aYCxGuy097bXeks6AuVCXHbh5qwy.SktssEToccwUhWfffM9lntZaiSW2DOu.wAaaL9BwAD5MjH_ DLihyO3csGlUuKRdQnFkakNe1_VJ88eTcltOEmEg_Y9Hb4zmMDqlxiaLjHBsMBToX4JLi_4B0o7v OFwFdhkQnD7VNmIPA.yg2md5DwbBHKev3ZHwKOdCgnNCk5RWOb8uTEfy6NfNUimJp_fxP.TndlA5 q803.ijRyNfxtnz.PdWRjldJ1SRcZFUDdI__qKjRh5DWk8qA7R8SG.7PUdTgyw4g0OxzCFKgrXIT er9f5xpvpo_0KS2W4m4C1yau7ywaHqnaiMK.Kh_zOvOP.j8tC0YtrdUvSrZUOruIETysQG9LzxWH eKzANgZKfGCFIMen_h8Rfqv1jF.IGaBQLo5MA26l0jIX4lJlLYZV_JQiwZKjGjYCBu4KShIrUQ3y FjeJix1pyC_6INdSB1P4.ODNqziWP0lqWluCA9KkCRJVn3dNbD3CGJ8Z1JdpGtfjEX9KYKO.8.oj zlQ-- X-Sonic-MF: X-Sonic-ID: a2a4f427-ebf3-4e62-bf88-6c0e93423d8e Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 09:24:09 +0000 Received: by hermes--production-gq1-6b7c87dcf5-rj4xx (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 770f63c3f5ca04d6e46eaf35c4227335; Sun, 13 Aug 2023 09:24:05 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: /usr/sbin/etcupdate -D/usr/obj/DESTDIRs/main-CA7-chroot -s /usr/main-src -M TARGET_ARCH=armv7 ends up building dtc ! From: Mark Millard In-Reply-To: Date: Sun, 13 Aug 2023 02:23:54 -0700 Cc: freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <058C6DE2-BB5B-46CA-A9C0-D50293137A8D@yahoo.com> References: <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2.ref@yahoo.com> <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2@yahoo.com> To: =?utf-8?Q?Mina_Gali=C4=87?= X-Mailer: Apple Mail (2.3731.700.6) X-Rspamd-Queue-Id: 4RNsY742kqz4YFS X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Aug 13, 2023, at 01:06, Mina Gali=C4=87 wrote: > from IRC, i have learned this is working as designed: > etcupdate without -B does a build. My interpretation of what "build" referred to for etcupate was way off. I've added the -B's that I was not using before. Thanks for the note! > Also, jrtc27 recently made it so bootstrapping, because otherwise = BUILD_WITH_STRICT_TMPPATH doesn't work: = https://github.com/freebsd/freebsd-src/commit/d81da4c98328d4ee3fe4c0a85f68= 74a3c69a1afd Yep: that added what leads to the dtc build. I still wonder if etcupdate builds more than it needs to for its specific pruposes when it does do a build: building dtc in order to complete an etcupdate just seems time/resource wasting. But, at least I'm using -B in the future. > Kind regards,=20 > Mina Gali=C4=87 >=20 >=20 >=20 >=20 > -------- Original Message -------- > On 13 Aug 2023, 08:32, Mark Millard < marklmi@yahoo.com> wrote: > I'd noticed that the etcupdate part of my build procedure has been = taking much time. So I looked with a ps -axldww while it was going on. = An example was: `-- /bin/sh /usr/sbin/etcupdate = -D/usr/obj/DESTDIRs/main-CA7-chroot -s /usr/main-src -M = TARGET_ARCH=3Darmv7 `-- make TARGET_ARCH=3Darmv7 -DNO_FILEMON buildetc = `-- make -m /usr/main-src/share/mk -f Makefile.inc1 TARGET=3Darm = TARGET_ARCH=3Darmv7 buildetc `-- make -f Makefile.inc1 _bootstrap-tools = MK_CROSS_COMPILER=3Dno MK_TOOLCHAIN=3Dno . . . `-- /bin/sh -e -c echo = "=3D=3D=3D> usr.bin/dtc (obj,all,install)"; cd = /usr/main-src/usr.bin/dtc;. . . install `-- make DIRPRFX=3Dusr.bin/dtc/ = all `-- c++ -O2 -pipe -fno-common -DNDEBUG . . . -c = /usr/main-src/usr.bin/dtc/fdt.cc -o fdt.o More may build. This is just = what was present when I happened to look. Seems an odd thing to be = happening for an etcupdate . I'll note that aarch64 targeting aarch64 = (self hosted build) also spends notable time and is likely doing = similarly. =3D=3D=3D Mark Millard marklmi at yahoo.com=20 >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 12:42:45 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RNxyb6qKgz4TjgM for ; Sun, 13 Aug 2023 12:43:03 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RNxyb6GPrz3LBy; Sun, 13 Aug 2023 12:43:03 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691930583; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LOFY2vifz95fCqD8mXLCA4UKyw09JFJnR5r0BE6WoJU=; b=YWU+nsoD2XS6whP0WiB0KlJo4gsOyQsDJTZiSE/eA7SiVm5pmqDbsl4SObI29CieoKhsuv EkZjPm2fbTtehRBVrxOq49/SA64b+wYUYzDTQvo+YMZ4b1f/EWz3G2FLj8HWYP9nfFPsfZ 2OgVvOrEUQv9Qd1qcyZ0GHbdPppgw7NY0oxj9MBd4fx9fA++2bQ12+k/K+eDndYDNCsN4E sIaSL/hmcjylOc2WtT+62/CZK7gT4StHFkOYtegsdhkNFcMESCN4sDzyVQRtJHVSyn6Cgx MzHpxelWoO9OCgVPCQDWgO8t0mkC1RvJ+HwrwMfWDWJRGciTAn/chhhgvxXDHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691930583; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LOFY2vifz95fCqD8mXLCA4UKyw09JFJnR5r0BE6WoJU=; b=eEHN0/+P6QqzgICBKrXEwAklowtENBRIHssuZHH5+rZDGOxhPdZOREP5cGiCJf/xTWuJjh OOuuiaI4dT7oYmWRttEtns57PeB9NaiqP8T14ZrflN4SNpTU52A3F858RivO0RiOfysp3Q IfqYX+1dgYxfapYDg+sJJHGcg3MeVBCSlM4zyvFDxXJHph/94VSv2rp3BZ6yTN4U5+7Now X5SewJY4IYyKBb6mxxWe1R6VWRn2o3wzR5uhj14IKPe8F7xog9UgRwWIqgQRDy3Jv+VBG2 M0JjeGpYTMUF9lLrX3glrufQI4LTgK0ukjW6fQj6F9efFMK0mPlVP67XUEjP+g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691930583; a=rsa-sha256; cv=none; b=EYNDlE+sp9W/YHXAekLVFkJQYXQYNXRWs+kXZ+Sn9VsTrADqovFgzrZmTvlWdGenlR38jr 0ko50T8SnRHRn2J+UfiePfVevsyDygkFD5Y6Oq4IsqZ8JY9j3E94b37eC4EQatH//Qm0rO YaxnnrkTXDrHvj0Ri6Qy1F8SdrIf094fzZ8anQkArLLlLrRHgNZLcn8c2yMM/LuUJqrXqx DPZo9oC5Sjz8LJFBbFOxInKCurYdl0EQS7kUnJASGwMszZxnQ5NnGWucHigRL1neJhuXKu +Z3sYBjRt2455EZYgRkPybADBsRwPtws9CE45nebnUbEufwUNf15ZyNVmBZK8Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RNxyb4Xk2ztxD; Sun, 13 Aug 2023 12:43:03 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [212.89.239.228]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 7030D61FAD; Sun, 13 Aug 2023 14:42:57 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> Date: Sun, 13 Aug 2023 14:42:45 +0200 Cc: FreeBSD CURRENT Content-Transfer-Encoding: quoted-printable Message-Id: <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> To: Mike Karels X-Mailer: Apple Mail (2.3731.700.6) Hi, > On 13 Aug 2023, at 00:17, Mike Karels wrote: >=20 > On 12 Aug 2023, at 15:32, Juraj Lutter wrote: >=20 > Did the buildworld start out by building a cross-compiler? >=20 > Have you tried without meta mode? With a clean objdir, I don't see = how > it would matter, but I'm not sure I've tried it. >=20 > The ALPHA1 builds seem to have worked, but I think they run on arm64. >=20 See here: https://files.wilbury.net/s/bJKKB5Kbta6DNqB for the full log. As I read it, lib32 is being built before crosscompiler. Is lib32 = supposed to be built on aarch64? otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 14:25:12 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP0Dj3Kc4z4TrP4 for ; Sun, 13 Aug 2023 14:25:25 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP0Dj2VvZz3TS6; Sun, 13 Aug 2023 14:25:25 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691936725; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SFu+VZPZwuQkBgsNPgPjJ+eRvdyT1EXlzJoAbpkV22c=; b=m8BwjNCcdJT5Y3T4M1VxorJ/IH3ig3PoWIXodQbgHxh8/pr4Q3EC+WBbzQeZolOWu1yLMN QqOzy3/tb3DQJcSgYOGzsS8uMZilscGRpTdDM9UY8R2MV+BVStQp8jW0sEj5HB+qgh3h9Q cyDe7kKzIWwUDJhsOxrPBIrlFPuokmpN3a004TTdhFFY+e/sVlDp9p9LOqWigSMTJGPpIu lFoq/cq+hmLrfiI076oo2/Pe49tNjj9Thk+/5z04dioWkYsOE5tQSkliJKDOkYjiZbasQY EFSTrh23yjZxxWekaVExTYJJbvg3nczJHWFQ2ZzbhBxnNFk2MhqWD4mqrgOu1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691936725; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SFu+VZPZwuQkBgsNPgPjJ+eRvdyT1EXlzJoAbpkV22c=; b=M35mjPHjCIdwS8TnuGXg9JSzbpXhQ2XK8QPmBk8lrbxxYUGX4MvpoxS7Z6W+/kvMI+9kj4 XGJpM1Z7uC+9a8I8a1Z7ZPdbePuY9fcZZZ13aoMrTbiKt/h+dQrWS2PO3Hs5LBUXiYywKP Ds/2aoZYKpyIoSTDV3eF7ea03ID5Yk1LF1AQpAqVBiCxZr2dBFrfs0cILHrY/+nIBEt7v3 e6ahrdijDRh2os7Qh2jy3WW1K8zCzJPVx/3WnaUrxUwnnrs4w1d3GqwsR3IZA9ur6gLtA2 OzocFpr9Vr3yWAmwJI4KeitAX0p9dtfA/JJZIZlIyrXEi72fDRVMbx0hAQksCw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691936725; a=rsa-sha256; cv=none; b=NrgbyCkfBoKeTbRqmvOyyUJg69ByA80CJru/WtHi6MKTaCttqKpxZDX3RHaWq7IK3XfwOV RZLVSW+gxkUaa+82VH6UdIkT9wDDTYVXOf/ovk2lT72ymcyWnxo90VKIdutlKvurNTdG/a YaA+1cP8Q40qjt0wpiJ8VDGpLFeANyLZqR99AMYMxyWDjp8teVPChTc7gAxaZKQ7OXf+ox TktfxsUqSEpHEpQ1VGm4eSKopofC7bQq1cJApP+8Eo/5yeRuuLDY3vzLTShjS1dFHUbcZn ziBSuEiKJPzwLH9jnI1xDppBqVdIWCnr8Ll53/RBH5w9K08uoY97Tb54itwUfQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP0Dj0mFxzx2f; Sun, 13 Aug 2023 14:25:25 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [212.89.239.228]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 46AC261F96; Sun, 13 Aug 2023 16:25:23 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> Date: Sun, 13 Aug 2023 16:25:12 +0200 Cc: FreeBSD CURRENT Content-Transfer-Encoding: quoted-printable Message-Id: References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> To: Mike Karels X-Mailer: Apple Mail (2.3731.700.6) > On 13 Aug 2023, at 14:42, Juraj Lutter wrote: >=20 > Hi, >=20 >> On 13 Aug 2023, at 00:17, Mike Karels wrote: >>=20 >> On 12 Aug 2023, at 15:32, Juraj Lutter wrote: >>=20 >> Did the buildworld start out by building a cross-compiler? >>=20 >> Have you tried without meta mode? With a clean objdir, I don't see = how >> it would matter, but I'm not sure I've tried it. >>=20 >> The ALPHA1 builds seem to have worked, but I think they run on arm64. >>=20 >=20 > See here: https://files.wilbury.net/s/bJKKB5Kbta6DNqB for the full = log. >=20 > As I read it, lib32 is being built before crosscompiler. Is lib32 = supposed to be built on aarch64? So it seems that =E2=80=9Cnative-xtools=E2=80=9D and associated targets = are not being built. If I do =E2=80=9Cmake TARGET=3Darm64 TARGET_ARCH=3Daarch64 = native-xtools=E2=80=9D, cross tools are being built and lib32 is built, in turn, also. The commit I suspect that lead into this was = eafd028327cee688b54bc526e088c2a3b98f94e0 (there sees to not be any other related changes fiddling with make targets). otis =E2=80=94 Juraj Lutter XMPP: juraj (at) lutter.sk GSM: +421907986576 From nobody Sun Aug 13 14:55:42 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP0vj1gl4z4TtZ6 for ; Sun, 13 Aug 2023 14:55:45 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail2.karels.net (mail2.karels.net [3.19.118.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "freebsd", Issuer "freebsd" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP0vh6g5Kz3YS3; Sun, 13 Aug 2023 14:55:44 +0000 (UTC) (envelope-from mike@karels.net) Authentication-Results: mx1.freebsd.org; none Received: from mail2.karels.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail2.karels.net (8.17.1/8.17.1) with ESMTP id 37DEthNv037286; Sun, 13 Aug 2023 09:55:43 -0500 (CDT) (envelope-from mike@karels.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=karels.net; s=mail2; t=1691938543; bh=BFw1jupS867EqWFHhLYPUKo+c8ZQ6TJoI1H452jTpe0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=VMcHy3Reu7YLLuUUpvk51le9TrPu/WQsKFUR7OaCPoMyZeYAPzs5Xb1z4fUblWKlq aliyAI24KitWjaVbMC8ucp07/9D6CG8hYv50E2w1JLoSWiZFTdmePAnR79vO5IjOzm ODwGJcpsIfFHR9uOreYicyHNXILXjJ1mwe9ge6uSWgr2NwME6uoxlUPNZDMQs9ZSD7 krL7Hp0Tmhf8KrmjNCPzK6yZimplcH9gR3puglw7dvfVIC+Xg2rembI7fv38/w2KaQ yjrZppJKsxA5Qjnw6ABMdL57g4Axf/affCrJ8p8ZkO6xNX8Iz0bFtOBgarfse9sg2u Tdh96NSbT9TqQ== Received: from [10.0.2.130] ([73.62.165.147]) by mail2.karels.net with ESMTPSA id yo2xH+/u2GSkkQAAs/W3XQ (envelope-from ); Sun, 13 Aug 2023 09:55:43 -0500 From: Mike Karels To: Juraj Lutter Cc: FreeBSD CURRENT Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 Date: Sun, 13 Aug 2023 09:55:42 -0500 X-Mailer: MailMate (1.14r5964) Message-ID: In-Reply-To: References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4RP0vh6g5Kz3YS3 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.16.0.0/14, country:US] On 13 Aug 2023, at 9:25, Juraj Lutter wrote: >> On 13 Aug 2023, at 14:42, Juraj Lutter wrote: >> >> Hi, >> >>> On 13 Aug 2023, at 00:17, Mike Karels wrote: >>> >>> On 12 Aug 2023, at 15:32, Juraj Lutter wrote: >>> >>> Did the buildworld start out by building a cross-compiler? >>> >>> Have you tried without meta mode? With a clean objdir, I don't see h= ow >>> it would matter, but I'm not sure I've tried it. >>> >>> The ALPHA1 builds seem to have worked, but I think they run on arm64.= >>> >> >> See here: https://files.wilbury.net/s/bJKKB5Kbta6DNqB for the full log= =2E >> >> As I read it, lib32 is being built before crosscompiler. Is lib32 supp= osed to be built on aarch64? lib32 is not built until stage 4.3.1, after build and cross tools. I tes= ted a build just now on amd64 with empty /usr/obj, and it worked (make -j$NCU= buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64). However, the host was approximately at the level of ALPHA1, and the first lines of the output showed that a cross-compiler was not needed. However, my earlier builds that required a cross-compiler worked fine. Yes, lib32 is supposed to be built on aarch64 as of a few weeks ago. > So it seems that =E2=80=9Cnative-xtools=E2=80=9D and associated targets= are not being built. > If I do =E2=80=9Cmake TARGET=3Darm64 TARGET_ARCH=3Daarch64 native-xtool= s=E2=80=9D, cross tools are > being built and lib32 is built, in turn, also. I don't see native-xtools built either. but the build worked. Have you checked src.conf? Mike > The commit I suspect that lead into this was eafd028327cee688b54bc526e0= 88c2a3b98f94e0 (there sees > to not be any other related changes fiddling with make targets). > > otis > > > =E2=80=94 > Juraj Lutter > XMPP: juraj (at) lutter.sk > GSM: +421907986576 From nobody Sun Aug 13 14:58:20 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP0yy38Zgz4TtGv for ; Sun, 13 Aug 2023 14:58:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-lf1-x130.google.com (mail-lf1-x130.google.com [IPv6:2a00:1450:4864:20::130]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP0yy1G0Hz3Zlv for ; Sun, 13 Aug 2023 14:58:34 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x130.google.com with SMTP id 2adb3069b0e04-4fe82ac3ab4so5182783e87.1 for ; Sun, 13 Aug 2023 07:58:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1691938712; x=1692543512; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=iPkFbWWLYeuEDhAkGvg5a6zogkexMby3+FTaKH8k2Cw=; b=XLGJrlv/6VtegC2QcN0pOmhghkjzGDfGbnNGstwAfSeY/S7VrnwkJKVSwfL61Re8jb winmpQgx3NsR2VRR2kZAudlT868CLixY/J+Yq1ZTS9x/JC96HL+Gzph0skNxdc706/Mw h43N7He50OrqjIRwb1pf/UyGQMkPpRTqLESpWlTHzPQAILO2+41LcgUAamtD3GmcITXN zqqEcdKuu+N01kbKELxPxsUm5mzY5nWv5/0vX9WvOnaY9P4EqbLe/MZUhAJ1Qe/uZK5r ufG7JIHOOiqbXUuE+y9aHm2MxKCJJmmmufvxctv/mZdoQPHxVrpOLtOw7zM+yRE+QxF/ jKLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691938712; x=1692543512; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=iPkFbWWLYeuEDhAkGvg5a6zogkexMby3+FTaKH8k2Cw=; b=drcNDTUojHhSia6mLznSgLLzY0pPN9JYNmAU/zZqKDUIqAcD5nM0gpHPzQdoqBvc1s ld3aAQ7gBrMmzUgL0qe5JxrM4VvMbAqtIm0fQbVdB0eyHeiJr9qYT9xGMYEKd6ICuvSX NtM1qz/693AMzVEbqYzzAKefQuo7br2ZQcv25MpY3kx4VXIId4ku9Wyvn3el1PMqUQCo KbijDU+63Y22Y1f9sgwdCPmMsl+ETDxMrT6XDgByboEnsIG7nM8WPkDdmfUC3n8HNtnk eSIFm2GF9Jm+q59UlYWo/AVE55YIWro0LxLNpmKuhi2gQQm1cGAMKrSowJkhXvNkpCaw fXeA== X-Gm-Message-State: AOJu0YyQQ5eu1vwshBtGtduwAwI2OjdtyfCz/thykJZnIBGTFsyu+7Uc w6helgq8uuJic8i1Pm7EVKYmyIRWpNYWaWp/5C6DGw== X-Google-Smtp-Source: AGHT+IHQnYohdMYVowBrc68pO4lEjCS73n3fTkdRW/9rMrIIlTC3r8l/CK0GBa6s16U9E9Xi2o1Y/M9uc7iZP+XUIOI= X-Received: by 2002:a05:6512:10d4:b0:4fe:25bc:71f5 with SMTP id k20-20020a05651210d400b004fe25bc71f5mr5299524lfg.11.1691938712015; Sun, 13 Aug 2023 07:58:32 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> In-Reply-To: From: Warner Losh Date: Sun, 13 Aug 2023 08:58:20 -0600 Message-ID: Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 To: Mike Karels Cc: Juraj Lutter , FreeBSD CURRENT Content-Type: multipart/alternative; boundary="000000000000e96c8c0602cf2d10" X-Rspamd-Queue-Id: 4RP0yy1G0Hz3Zlv X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000e96c8c0602cf2d10 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sun, Aug 13, 2023, 8:55 AM Mike Karels wrote: > On 13 Aug 2023, at 9:25, Juraj Lutter wrote: > > >> On 13 Aug 2023, at 14:42, Juraj Lutter wrote: > >> > >> Hi, > >> > >>> On 13 Aug 2023, at 00:17, Mike Karels wrote: > >>> > >>> On 12 Aug 2023, at 15:32, Juraj Lutter wrote: > >>> > >>> Did the buildworld start out by building a cross-compiler? > >>> > >>> Have you tried without meta mode? With a clean objdir, I don't see h= ow > >>> it would matter, but I'm not sure I've tried it. > >>> > >>> The ALPHA1 builds seem to have worked, but I think they run on arm64. > >>> > >> > >> See here: https://files.wilbury.net/s/bJKKB5Kbta6DNqB for the full log= . > >> > >> As I read it, lib32 is being built before crosscompiler. Is lib32 > supposed to be built on aarch64? > > lib32 is not built until stage 4.3.1, after build and cross tools. I > tested > a build just now on amd64 with empty /usr/obj, and it worked (make -j$NCU > buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64). However, the host was > approximately at the level of ALPHA1, and the first lines of the output > showed that a cross-compiler was not needed. However, my earlier builds > that required a cross-compiler worked fine. > > Yes, lib32 is supposed to be built on aarch64 as of a few weeks ago. > > > So it seems that =E2=80=9Cnative-xtools=E2=80=9D and associated targets= are not being > built. > > If I do =E2=80=9Cmake TARGET=3Darm64 TARGET_ARCH=3Daarch64 native-xtool= s=E2=80=9D, cross > tools are > > being built and lib32 is built, in turn, also. > > I don't see native-xtools built either. but the build worked. > Native-xtools are for poudriere and cross building jails. Warner Have you checked src.conf? > > Mike > > > The commit I suspect that lead into this was > eafd028327cee688b54bc526e088c2a3b98f94e0 (there sees > > to not be any other related changes fiddling with make targets). > > > > otis > > > > > > =E2=80=94 > > Juraj Lutter > > XMPP: juraj (at) lutter.sk > > GSM: +421907986576 > > --000000000000e96c8c0602cf2d10 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Sun, Aug 13, 2023, 8:55 AM Mike Karels <mike@karels.net> wrote:
On 13 Aug 2023, at 9:25, Juraj Lutter wrote:

>> On 13 Aug 2023, at 14:42, Juraj Lutter <otis@FreeBSD.org> wr= ote:
>>
>> Hi,
>>
>>> On 13 Aug 2023, at 00:17, Mike Karels <mike@karels.net>= wrote:
>>>
>>> On 12 Aug 2023, at 15:32, Juraj Lutter wrote:
>>>
>>> Did the buildworld start out by building a cross-compiler?
>>>
>>> Have you tried without meta mode?=C2=A0 With a clean objdir, I= don't see how
>>> it would matter, but I'm not sure I've tried it.
>>>
>>> The ALPHA1 builds seem to have worked, but I think they run on= arm64.
>>>
>>
>> See here: https://files.wilbury.net/s= /bJKKB5Kbta6DNqB for the full log.
>>
>> As I read it, lib32 is being built before crosscompiler. Is lib32 = supposed to be built on aarch64?

lib32 is not built until stage 4.3.1, after build and cross tools.=C2=A0 I = tested
a build just now on amd64 with empty /usr/obj, and it worked (make -j$NCU buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64).=C2=A0 However, the host w= as
approximately at the level of ALPHA1, and the first lines of the output
showed that a cross-compiler was not needed.=C2=A0 However, my earlier buil= ds
that required a cross-compiler worked fine.

Yes, lib32 is supposed to be built on aarch64 as of a few weeks ago.

> So it seems that =E2=80=9Cnative-xtools=E2=80=9D and associated target= s are not being built.
> If I do =E2=80=9Cmake TARGET=3Darm64 TARGET_ARCH=3Daarch64 native-xtoo= ls=E2=80=9D, cross tools are
> being built and lib32 is built, in turn, also.

I don't see native-xtools built either. but the build worked.

Native-xto= ols are for poudriere and cross building jails.

=
Warner=C2=A0

Have you checked src.conf?

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Mike

> The commit I suspect that lead into this was eafd028327cee688b54bc526e= 088c2a3b98f94e0 (there sees
> to not be any other related changes fiddling with make targets).
>
> otis
>
>
> =E2=80=94
> Juraj Lutter
> XMPP: juraj (at) lutter.sk
> GSM: +421907986576

--000000000000e96c8c0602cf2d10-- From nobody Sun Aug 13 15:00:03 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP10w6DMPz4Ttkg for ; Sun, 13 Aug 2023 15:00:16 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP10w5WQzz3bck; Sun, 13 Aug 2023 15:00:16 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691938816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gmah5nl0bZr5lCiw2/JpeCtJNUHdFZPoWtZEXVPv/rE=; b=B5TEQuOBUv9whEl+3qx7rfj7iBGqdUgtQ5CB4QU3hKH4b6LRu32CJ4xSCfuJnsEup2hpWL NUXZYEmRPPd+S0ZYKSCYugnQCDvdmHlYJjntAFJEAIB4w2Os1yx0z9kFkbCuq3VJwZa+yO ZO5JRO+mS8N/gZqPMF4DsM7UwIiQ0wcUJZ0CWtFZH6pvU28CfGLTisGaYxiUz5gJN5BAr5 CK4ZTwvmfKxEEcEVwzKmvx7f52XwN7fVF3vVNZJpAxgjy4e5P7l4uUX/19jb0gnJrx1JYB 7EG2/nJtDDXJlG9wrkd84paZkebDapNBI+fFM97dhn1UU86kRhwuSnAGM5g8vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691938816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gmah5nl0bZr5lCiw2/JpeCtJNUHdFZPoWtZEXVPv/rE=; b=dFvAHHNWqsqRa4YwEV5h92unyoj9zjn0kGe+QGNhUWVIn8CSURzqMdd0ms4J/tLboxMtC0 RhCf6NQM9hXiN51Y9KuiXgiCMalhiqByEs3GGBltGU8nGVaHnEdgq7JOsVERimbSVmHr2/ hsr+eexRtpFEIEBg5rtxpudbqkhw6BIpvG4N63oRj/V9XI+t9i9puqc8v+cp1R8VBvYqz4 2/Vlg3y+ddOxIaWV/+vj5x7N0ptjawHoVtcfBjroz61XrukebA8Am7fh1tEgWCPxxw2l7X 6nBgGVMlgkS7Avq8mn8aLxkj99/4vFxj9PzRBPl7yU0rNJioIO4pfREcvBsPyg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691938816; a=rsa-sha256; cv=none; b=r2ke4z0cuQJe3tQGBfriRrtjnT+qh/bNRHd9Txngbbt6gSoUfPSP2nVpcrrd5JoP9cnkzP QCNHrQBTEWmoRE4JbBBAtCmTyZCOqzlaEKp0gvEKaMe/pdH0HYT77bBVhSZ6WNpGAq4NfR n0SUXuqWpkjmxiLEpo1y28topHhd1AzludGA+WQJOVDqS2F/boxqSfSxRacOIurq+u9H/m degUf7NEackjZe5pMMu5e4WZU8NNUBJHWbhNY49pabwjaYsPcmMWBL4fv7zKkSYRNV61F/ PFkJb1VIj00f7bI8PPYYgMLQBYZxjR6B+j3LTNdl98kiLOprWxrRTv5SpHwh/g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP10w3lkczwjP; Sun, 13 Aug 2023 15:00:16 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [212.89.239.228]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 3A98761F98; Sun, 13 Aug 2023 17:00:14 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: Date: Sun, 13 Aug 2023 17:00:03 +0200 Cc: FreeBSD CURRENT Content-Transfer-Encoding: quoted-printable Message-Id: <81A42C4C-2A83-4D9B-A291-EBB360D06AD3@FreeBSD.org> References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> To: Mike Karels X-Mailer: Apple Mail (2.3731.700.6) > On 13 Aug 2023, at 16:55, Mike Karels wrote: >=20 >=20 > lib32 is not built until stage 4.3.1, after build and cross tools. I = tested > a build just now on amd64 with empty /usr/obj, and it worked (make = -j$NCU > buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64). However, the host = was > approximately at the level of ALPHA1, and the first lines of the = output > showed that a cross-compiler was not needed. However, my earlier = builds > that required a cross-compiler worked fine. >=20 Earlier builds worked for me as well. I use it for =E2=80=9Cmake = packages=E2=80=9D that I use to upgrade my RPi. It definitely worked on 20230712 when I did the build last time. > Yes, lib32 is supposed to be built on aarch64 as of a few weeks ago. >=20 >> So it seems that =E2=80=9Cnative-xtools=E2=80=9D and associated = targets are not being built. >> If I do =E2=80=9Cmake TARGET=3Darm64 TARGET_ARCH=3Daarch64 = native-xtools=E2=80=9D, cross tools are >> being built and lib32 is built, in turn, also. >=20 > I don't see native-xtools built either. but the build worked. >=20 > Have you checked src.conf? Yes, I only have KERNCONF there. I will try with = eafd028327cee688b54bc526e088c2a3b98f94e0 backed out to see if it will make any difference. otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 15:17:16 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP1NZ3Cdtz4Tvq7 for ; Sun, 13 Aug 2023 15:17:18 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail2.karels.net (mail2.karels.net [3.19.118.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "freebsd", Issuer "freebsd" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP1NZ2HrBz3dt5; Sun, 13 Aug 2023 15:17:18 +0000 (UTC) (envelope-from mike@karels.net) Authentication-Results: mx1.freebsd.org; none Received: from mail2.karels.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail2.karels.net (8.17.1/8.17.1) with ESMTP id 37DFHHtc037466; Sun, 13 Aug 2023 10:17:17 -0500 (CDT) (envelope-from mike@karels.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=karels.net; s=mail2; t=1691939837; bh=YJKrip/FgPmCrs6dOxm4Df0ms/4B97N9sMIFGUzdL7M=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=A7YMGQBa03HFyVJHNuTs0qZa9czisimzWSArChJxxqJXxqEELS5tat6EMLcQx2vCm hKFf3WsgVPp+Iog+CUl9Wlb92BneCfww8EMLf5mqNjII3Rzhy1/fjA/k9pMua8ypX+ jvKGy+5gGVEfe0/5q1E0xAtKLxkitANHScfFpgCjDc5i8brPB3MMzUTVggDU2JEFs2 IgQaB7IiUn+Cl0/azrt7vDs/WjB6qL6Xrz5X3nJ/IzAMuPWZIweDEbwhdjIlyyNd2/ eJgFGviq/gufzHcPpfDa4YOXIJ6RLtfQcNIhJ0HYFmsPriDvVLLOpwjg5bELHpSDNi rutN86zcS7swg== Received: from [10.0.2.130] ([73.62.165.147]) by mail2.karels.net with ESMTPSA id EpoyCP3z2GRYkgAAs/W3XQ (envelope-from ); Sun, 13 Aug 2023 10:17:17 -0500 From: Mike Karels To: Juraj Lutter Cc: FreeBSD CURRENT Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 Date: Sun, 13 Aug 2023 10:17:16 -0500 X-Mailer: MailMate (1.14r5964) Message-ID: <5B72C521-91B9-40AF-8FEC-8A8B687F6A3D@karels.net> In-Reply-To: <81A42C4C-2A83-4D9B-A291-EBB360D06AD3@FreeBSD.org> References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> <81A42C4C-2A83-4D9B-A291-EBB360D06AD3@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4RP1NZ2HrBz3dt5 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.16.0.0/14, country:US] On 13 Aug 2023, at 10:00, Juraj Lutter wrote: >> On 13 Aug 2023, at 16:55, Mike Karels wrote: >> >> >> lib32 is not built until stage 4.3.1, after build and cross tools. I = tested >> a build just now on amd64 with empty /usr/obj, and it worked (make -j$= NCU >> buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64). However, the host w= as >> approximately at the level of ALPHA1, and the first lines of the outpu= t >> showed that a cross-compiler was not needed. However, my earlier buil= ds >> that required a cross-compiler worked fine. >> > > Earlier builds worked for me as well. I use it for =E2=80=9Cmake packag= es=E2=80=9D that I use to upgrade my RPi. > > It definitely worked on 20230712 when I did the build last time. That was before the lib32 addition went in (20230725). Mike >> Yes, lib32 is supposed to be built on aarch64 as of a few weeks ago. >> >>> So it seems that =E2=80=9Cnative-xtools=E2=80=9D and associated targe= ts are not being built. >>> If I do =E2=80=9Cmake TARGET=3Darm64 TARGET_ARCH=3Daarch64 native-xto= ols=E2=80=9D, cross tools are >>> being built and lib32 is built, in turn, also. >> >> I don't see native-xtools built either. but the build worked. >> >> Have you checked src.conf? > > Yes, I only have KERNCONF there. I will try with eafd028327cee688b54bc5= 26e088c2a3b98f94e0 backed out > to see if it will make any difference. > > otis > > > =E2=80=94 > Juraj Lutter > otis@FreeBSD.org From nobody Sun Aug 13 15:22:19 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP1VS1PxTz4Tw3j for ; Sun, 13 Aug 2023 15:22:24 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP1VR3011z3gPd for ; Sun, 13 Aug 2023 15:22:23 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.57.232] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qVCv6-0008DV-SP for freebsd-current@freebsd.org; Sun, 13 Aug 2023 17:22:20 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37DFMJfX037713 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sun, 13 Aug 2023 17:22:19 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37DFMJTY037712 for freebsd-current@freebsd.org; Sun, 13 Aug 2023 17:22:19 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Sun, 13 Aug 2023 17:22:19 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: Fwd: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.57.232 X-Spamd-Result: default: False [-2.80 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RP1VR3011z3gPd fwd to freebsd-current@ because I don't know if kde@ is read by someone; matthias ----- Forwarded message from bugzilla-noreply@freebsd.org ----- Date: Sun, 13 Aug 2023 15:08:04 +0000 From: bugzilla-noreply@freebsd.org To: kde@FreeBSD.org Subject: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270041 --- Comment #11 from Matthias Apitz --- I stumbled across the bug in mid August again and had to wait two times in poudriere 6 hours to get to the failure. After the 2nd time I search with Don Google to get to this fix... The issue should be re-opened again (I don't know how to do so) and the maintainer should fix this port appropriately. Thanks -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ----- End forwarded message ----- -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Sun Aug 13 15:22:49 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP1WC10crz4TwNK for ; Sun, 13 Aug 2023 15:23:03 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP1WC00kQz4DSQ; Sun, 13 Aug 2023 15:23:03 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691940183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EcLimsvMT6MRnqtPQZcJAQ5gkOj98Q0TxeS4f7ZdaEI=; b=vzNG/VUfZ2mRm6egRVctGJjC6Nohwem9baXg8AACi5yHf0QlIoesmnFkYHi80u6Xte6vVD k7rRhrEeilyMSWWB/clcsDmbuZjedM7OOTq35HLH0QOMQaV5daGdUevvv1z1cN69VSTeIo c5iTc65R+MePKPLNrm8uLt+i9rTad9IJtFPM8buhLI2lG8181tdSXH3eeZpUXfHHHgQPx+ FZFJtLpeDUcEgQ25eE5JSAwDd43J//bxcFYzjtT4kuFBIqEclCrPe/OKDbBXcxaM0WvEbJ T1x9ZWOgY29NqUGu27WRh4EpuqLTDHlCRL4Us5VRAMTZJrXe4ATQpUI+nccuZw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691940183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EcLimsvMT6MRnqtPQZcJAQ5gkOj98Q0TxeS4f7ZdaEI=; b=c580CXVZzkvE8Xc75ky2ynoQ0LFpVIwkB1iKcs3qXHPjhRQmHMSbpk8zbu02w6TDmvoMZi 5vuWktFDQLGb7pVBrROUr6WEVdIisq9rcRKpJ5G7QrszqMwo/KGrkC6T0p9SLVBDF8+W5j LbotbtM/0YxWvRV06ABKfeROvYKGluTzvXEw+xhpo83VeLhl8X42nFwaMEAgalLL79bNnV NzWlSTQY9vUWebpErtcGn5KQ0nYl6cbq/KmTYC9qQ/fCITOw/TpBUuNl+6J8jZsrVJprHd WdiwPnnlEL4X86FLZqENAp1LKEcOJ8mzk79cjn/rdiVj1Jk+NKzB21xHosTL3g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691940183; a=rsa-sha256; cv=none; b=UbCUL9Wa/Y3gNMrs8Fkr1ABF/h9M31tNCKoxczzoCtUgRzJVJnPcEfzOrTmVt/A3BI4yU6 jwJXkvSkVyY29mH01wlyhEcJWo46U7Hy+lYhYhjQxxTClS5jYwhzvIMKbFfer+1LMS4qdZ 7WeyGaZYVRp0Z86JJ4dC43I7KsjTCiMqDPxwD5FvOiCCwX9psrtLZci/S9GZIFDwr32JZ+ IvCG/EVM6hRRzhZ0/ToxNm+KZUxql+UqPEMvxHY+F2MB5ERLfUL5H18GEAiM0k/rPaCpbx IKnyXv3My2DbS0ULMNs1CZu62bJbgd/hbb7s1QTwszROJQM/YESKoB4TizNgWQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP1WB5LkNzxKw; Sun, 13 Aug 2023 15:23:02 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [212.89.239.228]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 10F3161F98; Sun, 13 Aug 2023 17:23:01 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: <5B72C521-91B9-40AF-8FEC-8A8B687F6A3D@karels.net> Date: Sun, 13 Aug 2023 17:22:49 +0200 Cc: FreeBSD CURRENT Content-Transfer-Encoding: quoted-printable Message-Id: <98164D9F-0BAE-494D-B8E7-85C284A5C326@FreeBSD.org> References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> <81A42C4C-2A83-4D9B-A291-EBB360D06AD3@FreeBSD.org> <5B72C521-91B9-40AF-8FEC-8A8B687F6A3D@karels.net> To: Mike Karels X-Mailer: Apple Mail (2.3731.700.6) > On 13 Aug 2023, at 17:17, Mike Karels wrote: >=20 >=20 > On 13 Aug 2023, at 10:00, Juraj Lutter wrote: >=20 >>> On 13 Aug 2023, at 16:55, Mike Karels wrote: >>>=20 >>>=20 >>> lib32 is not built until stage 4.3.1, after build and cross tools. = I tested >>> a build just now on amd64 with empty /usr/obj, and it worked (make = -j$NCU >>> buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64). However, the host = was >>> approximately at the level of ALPHA1, and the first lines of the = output >>> showed that a cross-compiler was not needed. However, my earlier = builds >>> that required a cross-compiler worked fine. >>>=20 >>=20 >> Earlier builds worked for me as well. I use it for =E2=80=9Cmake = packages=E2=80=9D that I use to upgrade my RPi. >>=20 >> It definitely worked on 20230712 when I did the build last time. >=20 > That was before the lib32 addition went in (20230725). Yeah, that=E2=80=99s why I will try with commit just before = eafd028327cee688b54bc526e088c2a3b98f94e0 or with eafd028327cee688b54bc526e088c2a3b98f94e0 backed out to see if it = will help. otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 15:54:33 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP2Cp6n1tz4mJD9 for ; Sun, 13 Aug 2023 15:54:46 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP2Cn3phHz4J2g; Sun, 13 Aug 2023 15:54:44 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37DFsYVv025297; Mon, 14 Aug 2023 00:54:34 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Mon, 14 Aug 2023 00:54:33 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Cc: Matthias Apitz , Kai Knoblich Subject: Re: Fwd: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 Message-Id: <20230814005433.4d5897c3beee5b189e727b62@dec.sakura.ne.jp> In-Reply-To: References: Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4RP2Cn3phHz4J2g X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] On Sun, 13 Aug 2023 17:22:19 +0200 Matthias Apitz wrote: > fwd to freebsd-current@ because I don't know if kde@ is read by someone; > > matthias > > ----- Forwarded message from bugzilla-noreply@freebsd.org ----- > > Date: Sun, 13 Aug 2023 15:08:04 +0000 > From: bugzilla-noreply@freebsd.org > To: kde@FreeBSD.org > Subject: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270041 > > --- Comment #11 from Matthias Apitz --- > I stumbled across the bug in mid August again and had to wait two times in > poudriere 6 hours to get to the failure. After the 2nd time I search with Don > Google to get to this fix... > > The issue should be re-opened again (I don't know how to do so) and the > maintainer should fix this port appropriately. Thanks > > -- > You are receiving this mail because: > You are the assignee for the bug. > You are on the CC list for the bug. > > ----- End forwarded message ----- > > -- > Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 > Public GnuPG key: http://www.unixarea.de/key.pub Reopened as original reporter. ;-) I myself is no longer bitten again, as I've raised ulimit in /usr/local/etc/poudriere.conf. CC'ing kai@ here, who committed the latest actual "update" to 5.15.8. Unfortunately, bugzilla picks the maintainer (and automatically mails to) from ports Makefile. So kde@ is picked as maintainer here. -- Tomoaki AOKI From nobody Sun Aug 13 17:04:39 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP3mq4sVqz4mPHS for ; Sun, 13 Aug 2023 17:04:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP3mp2TK8z4Ttm for ; Sun, 13 Aug 2023 17:04:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=Up9eB2D0; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691946296; bh=vqOo9AzPLOgdkEcr022/HgLxSvskz9JIR5KJEJA6AME=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=Up9eB2D0qPTp9GJZm10T/8+TwwPfb/mwMKzAipVN6xO8LG1y05I8JyvYDjzFMxXGAtRUy3RZW2BRapHuUnfKKrLnXfvWXdNv4HBeBdvTBn0FUgOGJviocKh1HTNDzUIx+VjYcweMiZurmUc9y8yop8jcr4sOw3M/cYHzNtDrmNbZ+bK5fvjJuzdk7KeAnMVH+I/ZaXQbm41YtcV4oScP7W9GYxx+dpvDg30Tvgu5LvLjYZYvKhg7g8bN3kpfVSOsfhdiJmnbLyr2zW4D0rWPCSPYmS6EtHm9d+q99jiWLG0Mp57u0NeCkX5ihCqXv5tQ2yLnmBg+3dA8CcqpYa/UBQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691946296; bh=VTFz/nhQFJ3RHpB7xQ5DLe2XHFuqgP6l9ohsBKuNG7l=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=ZQnLsXdyQtmNpcWFlwM+hfCaZBl/DR7eN6VXdrFOCyrRfh29smOArslargC4JiGMM1ZucnzKiUilQM/qDsxIG9fUdmRrZCDwRRsoNAmVuTJKFwu451syGeAZTIXrjFiML1thX4KepFY1jSJzr3Mf4gp9tgT6AQGXnLqDQWulADkdIdm1KDRwgY2Vwv8wx50DB5muDqRtUFO7We4Ke2q4NxOeOrvEIh2oYAup9CIzvRdbPn4Ny+gfdCfbBTLYtLn3H09Xg2NTXKriGxU8f3i4i+1urevxlzYJuySLDehmhZcD24f1by9VWt1+tFfgb0zepuWgI2/a3fT5dD/u3XJw4g== X-YMail-OSG: JOARoMQVM1lVQ3YCUunbPnuJN6sHiweQw8O4edcJ6dOcHWvCjErf68opPnER6Db qdWOY03NOgqMeO31Wa.9j2O0pi4ioNZX0hRobP8VYoPDpTzAzs17ZG_d6DFkV.j5CpbEcebL84UP WFzaJ3s3B1M5QdO9ogbhqIrWCoEOLzUOQ5iRSdD1I4n16M2GWju7t8wusfYo_EgsIpLDz07gVRTI fN3uZctREKVDXlob7Vgt.YRxfxFpKl8jTkkC5r4j2PYss4rNqzdHW.NqZ4N8N0PDHaVn2TzfZUEn 6JrM4juRzA3dLCGbuXNu_Q5AmP2BWXLALF9ajo3OjC6dzfkAyVFTv_y6K8GSQbgDEwr134Fa_JDB hgjqWZ9qP2OBmGNWVK1UR9.8kX7taiMjccb38NRkVZPqnre_Tg5aO42DfzeVe_68JqD41OPFmpRi OVPXN8os7LyLPDLx2yAFCLOMRiS5XVm8NJHqvVTp0a6Q9bPaG0J.zHQdxIpP5V059aBtkS2kRrxp XTCXd7KuveEQptLslFafVhm9mKNcFBV_srrTDB5qNv.qGt6kGdQRFtiv7yKJ1A5liL4g0aQFJ9rA EnZdapCqSBKH36o5Abm93Etej3fHj7o1I4DgZtOlhgFikyObGpjclIJcyTws7bL1ZzNXGRxRYFZd IRDzKv0TwXwEy8bose5KS_Ohrl4pxyHGPlRjC.yAHfmOvdNO9B1VcduBFEQbDQsvNOYORcYx_i.l Xqr1JtdC6q19POhG4NCnRY_kxqUjbYx2RVPX5couA34pcQrEdJQPcrfLmC6I7_rcd1jKQfuVjH45 qAHL8M_kK0PJbK9q7v3ALYJyfVNYjteGjj4S6xm7ncN2et0cook22EsHNNnhUfctzW5keRgaJRKY d6U1tUnA2oZuFb.7R9yPhni8hpVkllbIKLthvngX6KWWrXv0RM5Z8QCAEs41FtFofK.gbZwsI3o0 MG.EROb2s9rsqDoAizBf7IhJPlyCjl3chgj5f7rOXIIN9ItcZCzKSzOyq8AZvYOqSFhQWS09V_VO qfJX4zLwdgzkW3MbCEbbs53CsOGq5wf167QFJSbfOb7Vxc0JxNriDJqmmb8nWEccOVc4SiijJiZ2 i8N1gzzKWe.Y_nA7woRBorz6RejIELau1qw.QhOtt8sj8GFz3QRxLyiZAQlS6F6JuGVTBOfcdGJ6 ZM6h57WNRinspV8sZ4avzgpfl4D9u7kpR.poKS6D1KrsRis9U_P9aMWS_9kmi_zHhMa6P1GRPFma Gzg.0Yth3piEPBJNLtd6i.golS6vpw7Mk2351fjAtWtQ_5W3ZMUFOf73ljC.ENiqjpqrZ8KLCWf3 cg96ighF_aZhr5MZMzK2qv006bmIuTZZYNLxHdBvIwKZUfsIJke38BZZVbcAPAM8OXNWDdju.6ss n3CbZWhkpSry6QnJWYbxLhhPxf0DbdgDgrOaKVkNeq6PR.HXFR.hstxt036NVYVDx4Kd4FP.hSVI S8caPQTHJwTeNLjSMkYnRTxaSUlRRRrW_p7BI.cptNp0JWvz8gnloQIAmMic0VdzmY0wG.vicihl vnmR06qHvm8zEGhE8Qb6gLV2ceBOc.4DpVkmBNXVoSuKMIfe83xnmLIznQKAUllnbhaGoujO2Ka1 vSknRuwnCHIw7Cy9puXyBmQ6jfNAlqj2MNIQQX6MNe3nYcjTxEdUqw1b5KVZnQXYMePGo1cEOeU3 9GFJiHpBnuXYWvirOIW8GcoeKJ6cwav1hyU7f_UZSGC5koTvLNpAocrGjU6NF6Kqpe3ORnH3HF3s ux_ypouNBXEBQ8cz63gGLGngF0Wcw83sY4eX_5rLDPp97mh2PwRNONMFfJ0UvjGCddkou.y1divB D4wREGeNgV_nq85rowZ6llSk5KL0td8L7uPULHlWyinLiSztvR8Zl.vvdiEBh6W8crGxTPZwWWic 2KRPiWDYWYIbroEm1E0rIPPJS0aUb8CgQvSF.uSD4HjCLlI8VWUHMotZVKpaAchr15FroAJwYL8B vop6X5VgQB.Cz2mqkRR2IXhEeItEK_QhCW_4fGZpgtllVJgtn0d2hKsLDjvsEIwMuIiouvAkSKN3 t2Ho0xOA6T5OTXkjfxLxWTFkrkz5GQLvVBMAv_Q7ffzWDau44bkCQr2bFA4Gv_G9MPkcXLcQLaXp UQs.eh8ONku1TB8z3A2IOHjMmIh0WjQT8HHFK7PbEw428fYEPWlkpA8DDPHuEvzTwqFEILKCBnxu EnIbPWHwh4xYZtF8LUi_wYkxyW3LHHgMR5jmEZty4kLb9SQ1yLsICFTtut_FiyfB1pr.2xBQNocw XuS.Y X-Sonic-MF: X-Sonic-ID: e46d25d3-791d-40d9-97af-ac49171acb8f Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 17:04:56 +0000 Received: by hermes--production-ne1-7b767b77cc-swgdr (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 53e30fc3496575d0b01ebcffb3b5a25f; Sun, 13 Aug 2023 17:04:51 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 Message-Id: <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE@yahoo.com> Date: Sun, 13 Aug 2023 10:04:39 -0700 To: Juraj Lutter , Current FreeBSD , FreeBSD Toolchain , Mike Karels X-Mailer: Apple Mail (2.3731.700.6) References: <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE.ref@yahoo.com> X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.993]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.147:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.147:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RP3mp2TK8z4Ttm Juraj Lutter wrote on Date: Sun, 13 Aug 2023 12:42:45 UTC : > > On 13 Aug 2023, at 00:17, Mike Karels wrote: > >=20 > > On 12 Aug 2023, at 15:32, Juraj Lutter wrote: > >=20 > > Did the buildworld start out by building a cross-compiler? > >=20 > > Have you tried without meta mode? With a clean objdir, I don't see = how > > it would matter, but I'm not sure I've tried it. > >=20 > > The ALPHA1 builds seem to have worked, but I think they run on = arm64. > >=20 >=20 > See here: https://files.wilbury.net/s/bJKKB5Kbta6DNqB for the full = log. >=20 Looks to me like "make package" is broken for context in which: make[1]: "/usr/src/Makefile.inc1" line 340: SYSTEM_COMPILER: libclang = will be built for bootstrapping a cross-compiler. make[1]: "/usr/src/Makefile.inc1" line 345: SYSTEM_LINKER: libclang will = be built for bootstrapping a cross-linker. happends. (Or "make package" just is just not intended to support such = contexts?) Details follow . . . Your log shows: make[1]: "/usr/src/Makefile.inc1" line 340: SYSTEM_COMPILER: libclang = will be built for bootstrapping a cross-compiler. make[1]: "/usr/src/Makefile.inc1" line 345: SYSTEM_LINKER: libclang will = be built for bootstrapping a cross-linker. But searching shows no such builds happening during arm64.aarch64/tmp/ , = just some of the prerequisites ( tools/lib/clang/libllvmminimal tools/lib/clang/libclangminimal = tools/usr.bin/clang/clang-tblgen tools/usr.bin/clang/llvm-tblgen tools/usr.bin/clang/lldb-tblgen ) ~/Downloads/bw-arm64-aarch64-out.txt:1: [Creating objdir = /usr/obj/usr/src/arm64.aarch64...] ~/Downloads/bw-arm64-aarch64-out.txt:21: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tools/build...] ~/Downloads/bw-arm64-aarch64-out.txt:48: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools...] ~/Downloads/bw-arm64-aarch64-out.txt:51: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/tools/build...] ~/Downloads/bw-arm64-aarch64-out.txt:168: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libllvmminimal...] ~/Downloads/bw-arm64-aarch64-out.txt:170: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libclangminimal...]= ~/Downloads/bw-arm64-aarch64-out.txt:172: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/dtc...] ~/Downloads/bw-arm64-aarch64-out.txt:174: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/fortune/strfile...] ~/Downloads/bw-arm64-aarch64-out.txt:231: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/libopenbsd...] ~/Downloads/bw-arm64-aarch64-out.txt:248: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/rpcgen...] ~/Downloads/bw-arm64-aarch64-out.txt:272: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/yacc...] ~/Downloads/bw-arm64-aarch64-out.txt:362: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/xinstall...] ~/Downloads/bw-arm64-aarch64-out.txt:387: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.sbin/bsnmpd/gensnmptree..= .] ~/Downloads/bw-arm64-aarch64-out.txt:415: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.sbin/config...] ~/Downloads/bw-arm64-aarch64-out.txt:433: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/vtfontcvt...] ~/Downloads/bw-arm64-aarch64-out.txt:477: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.sbin/zic...] ~/Downloads/bw-arm64-aarch64-out.txt:487: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/mandoc...] ~/Downloads/bw-arm64-aarch64-out.txt:545: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/localedef...] ~/Downloads/bw-arm64-aarch64-out.txt:637: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/mkesdb...] ~/Downloads/bw-arm64-aarch64-out.txt:717: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/mkcsmapper...] ~/Downloads/bw-arm64-aarch64-out.txt:750: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/kerberos5/tools/make-roken...= ] ~/Downloads/bw-arm64-aarch64-out.txt:805: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/kerberos5/lib/libroken...] ~/Downloads/bw-arm64-aarch64-out.txt:1041: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/kerberos5/lib/libvers...] ~/Downloads/bw-arm64-aarch64-out.txt:1056: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/kerberos5/tools/asn1_compile.= ..] ~/Downloads/bw-arm64-aarch64-out.txt:1122: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/kerberos5/tools/slc...] ~/Downloads/bw-arm64-aarch64-out.txt:1153: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/compile_et...] ~/Downloads/bw-arm64-aarch64-out.txt:1263: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/clang/clang-tblgen...= ] ~/Downloads/bw-arm64-aarch64-out.txt:1265: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/clang/llvm-tblgen...]= ~/Downloads/bw-arm64-aarch64-out.txt:1267: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/usr.bin/clang/lldb-tblgen...]= ~/Downloads/bw-arm64-aarch64-out.txt:16253: [Creating objdir = /usr/obj/usr/src/arm64.aarch64/release...] No arm64.aarch64/tmp/ SYSTEM_COMPILER or SYSTEM_LINKER bootstrap = compiler/linker were built. =46rom what I can tell, this leads to: cc -march=3Darmv7 -m32 -target armv7-unknown-freebsd14.0-gnueabihf = -DCOMPAT_LIBCOMPAT=3D\"32\" -DCOMPAT_libcompat=3D\"32\" -DCOMPAT_LIB32 = --sysroot=3D/usr/obj/usr/src/arm64.aarch64/tmp = -B/usr/obj/usr/src/arm64.aarch64/tmp/usr/bin = -B/usr/obj/usr/src/arm64.aarch64/tmp/usr/lib32 -O2 -pipe -fno-common = -fPIC -g -gz=3Dzlib -MD -MF.depend.libssp_nonshared.o = -MTlibssp_nonshared.o -std=3Dgnu99 -Wno-format-zero-length = -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k = -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes = -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch = -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts = -Wnested-externs -Wold-style-definition -Wno-pointer-sign -Wdate-time = -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body = -Wno-string-plus-int -Wno-unused-const-variable = -Wno-error=3Dunused-but-set-parameter -Qunused-arguments -c = /usr/src/lib/libssp_nonshared/libssp_nonshared.c -o libssp_nonshared.o using the pre-existing system compiler that, in your context, gets: error: unable to create target: 'No available targets are compatible = with triple "armv7-unknown-freebsd14.0-gnueabihf"' 1 error generated. I doubt that "make package" was tested for being able to handle the lib32 addition to aarch64 builds for SYSTEM_COMPILER or SYSTEM_LINKER bootstrap cases. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 17:10:16 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP3vB1fBKz4mPcq; Sun, 13 Aug 2023 17:10:30 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP3vB12x7z4Wln; Sun, 13 Aug 2023 17:10:30 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691946630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OghbVaY57ZsxDuI5uyx8vsiIJWLBIvGm0mpyBOyTh4k=; b=KO30AwqArNK5FbyLxZtkwS6jkM+MUuQ9LT+P6LktNeBFIy6IR3HSSeP2h/VV6FNhK/R8f0 oi0tOsZaiuzKXGDGUMNgnIxhlrpuMm/KWlMRd2RPhOwZfw3xuXZIB4Mo6TUm0rRfvCBDpq hINT0ZdJBg/LGNyoKgTRw8hAJA9DH/HgM6Ig1ICw3/z5gkH16mWiA5C1CRpjtNSRCqJ6mw NWcxl9i13Yb4rY01SdGS8KYUXGICCM5gipYQE32qO46PfKTGV2xdN0FHWQ0bdFzgc1XSqY mGzw7YAZ6MyppmKWMDXG6grx4+JXVGn2cS6gDClloG9AVmO6kWkwJY7kkfqzzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691946630; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OghbVaY57ZsxDuI5uyx8vsiIJWLBIvGm0mpyBOyTh4k=; b=tUfu1bG3O7OokNwHYyHKDaBwIobIvXiHrTl4Dy0zjdA4ZmU8gkKLbE8bmxLQMg0B8K090J AqSryrIltqblcZ61qrZuHNJyzqx4m5UlY7rY//hD1y1mZHfnQiOdLtCV6oBfzr7DgPdN/5 rH01ttxoqxgkMFdRhrtnvZS6XJum0RjJDheDN+gYFL7bUUMYCI3DNGM3WIivssQwp1te+f PoIbv/aTApiMa6tnEOZ5Bs1fw7G/yMsrKEYLZSHUs18+X8AuN1xZbzit4MgOgDBOYnol3F pfNWFjdRBZPsYN0Bxz6Eb1Wq63ml4sWy8eq+wHqBF34Elpk099Mp/xn+oV6OSA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691946630; a=rsa-sha256; cv=none; b=fwTiQw6jVHBCfgkpKf5AA+osf1lFs3Snn1JfJ3nxAY7MsXGf9fZgtBlFgc+bxwrMwAoOr9 Buq8tdP4w4AaFNUgKOJvK4lQPCG4aRZb45KTZHQqCIS+/h23c4hG3avVaLFj6EQEnUYLfM rdg8lmBWj5T+pjRDERf837W2wR32vJASeYdSgqCb37fL+3V5e88T/N5D+KkY/gG6cdaA3h AtZhGn+qTod/lhwjvZJhvitPA+3R+4AWpsjxZgFIHXeudNjhvP/Aa6CbhXBGPN+57w2ZCy AGhnHT+4O+jnP7zhtfJQ1nUIqTPYOJEA6ok+Gzxei0Ntnk3rf4VxtUqtiug0SQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP3v96NTgz10CS; Sun, 13 Aug 2023 17:10:29 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [212.89.239.228]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id D8E2261F96; Sun, 13 Aug 2023 19:10:27 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE@yahoo.com> Date: Sun, 13 Aug 2023 19:10:16 +0200 Cc: Current FreeBSD , FreeBSD Toolchain , Mike Karels Content-Transfer-Encoding: quoted-printable Message-Id: <3D94C0A3-0231-4C04-8C69-5516A4A41B03@FreeBSD.org> References: <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE.ref@yahoo.com> <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE@yahoo.com> To: Mark Millard X-Mailer: Apple Mail (2.3731.700.6) Mark, > On 13 Aug 2023, at 19:04, Mark Millard wrote: >=20 > No arm64.aarch64/tmp/ SYSTEM_COMPILER or SYSTEM_LINKER bootstrap = compiler/linker were built. >=20 > =46rom what I can tell, this leads to: >=20 > cc -march=3Darmv7 -m32 -target armv7-unknown-freebsd14.0-gnueabihf = -DCOMPAT_LIBCOMPAT=3D\"32\" -DCOMPAT_libcompat=3D\"32\" -DCOMPAT_LIB32 = --sysroot=3D/usr/obj/usr/src/arm64.aarch64/tmp = -B/usr/obj/usr/src/arm64.aarch64/tmp/usr/bin = -B/usr/obj/usr/src/arm64.aarch64/tmp/usr/lib32 -O2 -pipe -fno-common = -fPIC -g -gz=3Dzlib -MD -MF.depend.libssp_nonshared.o = -MTlibssp_nonshared.o -std=3Dgnu99 -Wno-format-zero-length = -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k = -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes = -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch = -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts = -Wnested-externs -Wold-style-definition -Wno-pointer-sign -Wdate-time = -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body = -Wno-string-plus-int -Wno-unused-const-variable = -Wno-error=3Dunused-but-set-parameter -Qunused-arguments -c = /usr/src/lib/libssp_nonshared/libssp_nonshared.c -o libssp_nonshared.o >=20 > using the pre-existing system compiler that, in your context, gets: >=20 > error: unable to create target: 'No available targets are compatible = with triple "armv7-unknown-freebsd14.0-gnueabihf"' > 1 error generated. >=20 True, exactly the same error I=E2=80=99m getting. >=20 > I doubt that "make package" was tested for being able to handle > the lib32 addition to aarch64 builds for SYSTEM_COMPILER or > SYSTEM_LINKER bootstrap cases. >=20 I=E2=80=99m getting it in buildworld, not in packages (it does not come = to packages target) otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 17:49:16 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP4mC0vXbz4mSbC for ; Sun, 13 Aug 2023 17:49:31 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP4mC01dxz4fgq; Sun, 13 Aug 2023 17:49:31 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691948971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=em3xvksJqSxFbKXnOiKrs1fysx6C8Nk6VFDdAYXCXto=; b=h3Iv7SVOfgDY+qmBVpTw+JT8GPXsozrMqCRIu8P0aPhU71/tzxJPDXCrgWGPe/ozTQ6uyX tFkpgyUiHt7j5h2ZHO3V2HqnIVJ/uXYfDuCxOABmKQduuzLKRDtpi5yvalbPMnhLk9uN52 tEzjqWADEwUUYXMyCM9rJIKEVSG4wDhSgng1fKA7R3YtwytLHg+/AqqnlUcSkzSMMnyPfP qFTdCGEH6CKSiPhpNkwIuoW5gzquYlq2CylewtvNXMVp3t09gB+lltwWIx0LelDEdmHESA oy+iLjDRX9jXbyywIqB1twmXtKw/DObw6qGMRGVImUeC/3gBzJ3RSrszngd0zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691948971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=em3xvksJqSxFbKXnOiKrs1fysx6C8Nk6VFDdAYXCXto=; b=alVTwzerqwmgIotu5PXxh0kmjbncSq9C6g2gfbpgL+JVeroWxaznK42QSSo0oFgEWY+zDL C3uuxxDjqdUsKgyiL2cIF0HEwlGnqKis8hsPU1V+CpvKfGszGhvYBsaO3aSxSVvmk/dLBu +/U7ekvXDr45glLuzBL75g2PNOxAQFbkPt5tGqelQ1esww8NMgI1uetBnFyMoVIKZuvhEq b/KUYlKah9FdhmpzGG79sy1cWYmku9lViJq8jV92bLnEW0seIi2EeAQwUg3Mwt5CA50Jsv JcPtb2xteYMTAQDHQb5gLe60capciM7aK+ELBb0yd38qdinCoyD0JBXnTzr+9w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691948971; a=rsa-sha256; cv=none; b=UOXe4A8iz8WfludeqQN+WkLSXlv+OyI1aQihiUuvvVy+uZiHMERfpvyD0RYvFhaY0ErxC1 ETQIcDKIkRI6QSzLmcyjf3tO4An/Pqe249ijJVd5FpZFbsE7uM7Aa4lhaNAAe7/ZZYezQ8 5Y2bFckG0XFyLfI/OroIdxUQXSgz7D8XNvVlnmnHPjWb+b1JfruiIEuk3zPXfrPI+NwpUZ cIF/sKfqr03XKN3xd+G7whxUsTHjjvbMmAJKUveiiw7XiqEKNfnIP090Ozq9ppQHHt4ZQW AHcoceRwVhS99xc1Kd+lVgdBKHwCKBHTthzWdi+EbLTptbnL89pkxaIzOgEOYg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP4mB5M3jz11Gp; Sun, 13 Aug 2023 17:49:30 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [212.89.239.228]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id CB76C61F96; Sun, 13 Aug 2023 19:49:28 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: <98164D9F-0BAE-494D-B8E7-85C284A5C326@FreeBSD.org> Date: Sun, 13 Aug 2023 19:49:16 +0200 Cc: FreeBSD CURRENT Content-Transfer-Encoding: quoted-printable Message-Id: <756857B2-01C8-4BFC-A607-33527C7E6E3B@FreeBSD.org> References: <68CBFC44-9C0A-43D1-A0C2-530207E0762F@FreeBSD.org> <0FA5C2F2-6639-4522-AA29-537F596A5F81@karels.net> <8A772467-B246-4C6B-AD7E-81AD16903196@FreeBSD.org> <81A42C4C-2A83-4D9B-A291-EBB360D06AD3@FreeBSD.org> <5B72C521-91B9-40AF-8FEC-8A8B687F6A3D@karels.net> <98164D9F-0BAE-494D-B8E7-85C284A5C326@FreeBSD.org> To: Mike Karels X-Mailer: Apple Mail (2.3731.700.6) > On 13 Aug 2023, at 17:22, Juraj Lutter wrote: >=20 >=20 >=20 >> On 13 Aug 2023, at 17:17, Mike Karels wrote: >>=20 >>=20 >> On 13 Aug 2023, at 10:00, Juraj Lutter wrote: >>=20 >>>> On 13 Aug 2023, at 16:55, Mike Karels wrote: >>>>=20 >>>>=20 >>>> lib32 is not built until stage 4.3.1, after build and cross tools. = I tested >>>> a build just now on amd64 with empty /usr/obj, and it worked (make = -j$NCU >>>> buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64). However, the = host was >>>> approximately at the level of ALPHA1, and the first lines of the = output >>>> showed that a cross-compiler was not needed. However, my earlier = builds >>>> that required a cross-compiler worked fine. >>>>=20 >>>=20 >>> Earlier builds worked for me as well. I use it for =E2=80=9Cmake = packages=E2=80=9D that I use to upgrade my RPi. >>>=20 >>> It definitely worked on 20230712 when I did the build last time. >>=20 >> That was before the lib32 addition went in (20230725). >=20 > Yeah, that=E2=80=99s why I will try with commit just before = eafd028327cee688b54bc526e088c2a3b98f94e0 or > with eafd028327cee688b54bc526e088c2a3b98f94e0 backed out to see if it = will help. To me it seems that f1d5183124d3e18d410ded61e45adb9a23b23c83 is the = commit that broke it. I did =E2=80=9Cgit revert=E2=80=9D on that single commit, these were = rebuilt: Building = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libllvmminimal/Supp= ort/Path.o Building = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libllvmminimal/Supp= ort/Signals.o Building = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libllvmminimal/libl= lvmminimal.a And cross tools started to build as well. otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 17:50:03 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP4n818hSz4mST5 for ; Sun, 13 Aug 2023 17:50:20 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP4n74F36z3Cc6 for ; Sun, 13 Aug 2023 17:50:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691949017; bh=+26y6wO7CYoL4i6O7JQ6A1XwA2Nz8ZRKlBW0/XrBe2s=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=UPpGMXp8SD5LkEK5l95LVZUdhnffd+OQ4l8VMWZDo/YKRHrEdfaknvp+ep/cCV3m7Ph4V81aEpMsKCMCkByH4FMb5wQaKjpC7VXt6hSwJzG/yRfqgdTcPG+pQ9K4q5mU5UnUxu74dmrfu+LeQr1rTPKyaWjIByJ9HD1g/t06AT21XfJl4t5T6UQmEQuMUuwrg9+fj0L8d12eN2lubRcQt/NawE5z+fF64QIp4jhipMKjkd2tMPemnMrYCGZLkM8awQKCYbB1avdaKOXtvFTeKiqhNYi937I2R+tMxv5ii6EhAEoXtGvKu1gu6rfoSUHsCqCE7FAM7P7W69/CperaXA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691949017; bh=lNfor5b+0xo+TMyR4b1QYlLROXTsTYzeY0gk8UDGEdZ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Y1wMpy5hY29y6FFcJxPeAJ66A48EBEa30jJqVXEsbwCTqTDTpIVvNazrgo9PEOK7OemenfRwC3R9qY5DI/b1/dcWQNnBiILMrfL348pbUkkffc4CrglA21NeA/MKz5TGyN+X33dh2vkV72bc7P65ujW3tIFlwW2U5MzBlQ9nxQeTzGjGolyi462JicCF4azaV2uhBSOKX92ZmBH78HB7Cp2TOC5I5dI4ovDW80dhoacUz80Kg5+NmbBqQQY6+66ucRfgqb0vDMXbAj0mG8QuVHiu5MgyiTo8CdT+pXcqr02RyXfLTW2tw05B+kAx8TF+6whsThTeYXTCX+Q7MtZ/IQ== X-YMail-OSG: EtuGa2wVM1lKUqt_SzBJRSq1FswkPn83Ek2It8DzpghAUicR1R6dsS8omvNkJez yYyZusC4eFzmfhnO3I0s4fvnK0Xprmq3Su9Qf60ZTnn9rycPcZ8zkbrZxVVoHhm1NYysewiUQab4 sy5nvLJujw0EvVS0NtLfczMqfAZPX201TseR4X__2PhVBKlzdwD.l2eTY64hC95_xxzC9vc_Fovk XMOT0Wf1SGGYuQdPTo9FhrVd0g0Te0appHBnmM39W5VnYY6ggVTMkpEMDkQvGN3UxO6Ni8ayi7P2 ML4Xg4RWIg.QkK2KcLQobSUiWYGbrnoRch3jrE5geSHJ58zMMeV4gDFnKyoJwYBYHKEj73tvxJrY mSyGAXU5isuYvD7IIi1teH8X6fvewflHKqXgSSMNUlgfTz8O5nkazBot5Nuln7njzx21qxLjowdP 3LFb64EcWk1AvQy_9S3deW2tcWZQBQWKomoZLIvMP4g6iOjgx3gWwwostFrLpkzRGL9wGN7Aelxg dWKC0hRM2wQq_X965WqWPC1Z_jwomKAEOVHfgVDezt2zQbdKOQIICzXIzRZxmP5hGWfqYRrOPAU7 0PAVuiw17yvtq.WCU2KAK09qqBvgbYUrxFDnrWcllw52iNQAAB4.Pjy3KQDiiA0SKffn3dny9KSe GJIn0kdJ8FLxLjiJEaCj6LY32rFkFqQ_N73jrMfpOB60TY0W.6W8yVdPAKaxarvEJf2E7Pw1S_rn eOgC5CCqubtemzwsw2M_KzitVEYICl9wE4eer6Dw8XWPgy84G3ImA4duIqNWYGRP0Mf5oc0Hr.83 dPTFdbHlbCu27t2cm2LAkbM_Vh3IFk_I9Czfheh.ouih7KGLFN6.0Ah.4gbCE88n09I2GgJtnInt cc6QdWJpJT4MbxdI08Fx3NXGgrjsP1zWYHwTt92brkWUmySZbPfiLZQe0.brdW5ryQ6wlYjNF7AL 4GkvNQSYS6.gRgYI7Zu.HEO6yptSQqHj32KovB3yyuHHBm3MF4dbpIVRt0WGlj_Wj7.IBqLR2Jrs X9000ZcYiiE1p4Gg1qAKJvDtxogB5nKpUfAgIVOYWqJFKv5Whd_BQ2V25eTZChiV.yqeo0SvShxS YQ3D17ry9hndXtdv_Vpa77wkK52ZoAxKMwA2maIr38jyxFFvMi6vMr8EYHL1mC6dO4zIyo93Xk7e hMwOIKWjgtMdx6z8vRthqiqCCSeoFV013kQRWsetxjDzvs.FcwP5_ML16JwqeetAbZEnrJ7zGwq_ _mSVbyyr2ZuLiuBqe0rZt5BUlX_YrAFsY1WIME_Rh.eaEqf3PHbI1gk2epEsTuySSJEOm8gSd8mZ X54uCiz2EG3VQTKRyuD5YH7CapYDUm3DkJtStsRrh4Su6wnjNEmdrCIP5IItH0ELOvTXXq0yd1IM kN8NpRqyFdkAFvDxZMIRxbie2IBVRdOcwpAJ75ynPgnoFAUUFKCv4KrAq5F8ffbxxdS0z1kADJos B7rnJn5vfPC7A7qsEIifn.vhQ505wPhMtNwnv4_Y2eo_wDWDd6G0uJDjem6rIFRebfoKhjw9Cd5G dEXqtVoOamBRHUfTWXFHUyauq9IPmBAhe8PPNQsxtpRVu8zETEneNsnAZyaGxs0Qie88b6jog9ii jvJgR8GjAq3eTHHa9ozvMi0BdRUSsBMn7j4sEmTS6bqAh2Bfoo9UHt0aZcDvBffmabO.to7LAyI9 wxP1J31W9PqospH2ylQOvDqDtdVHZqXxcLDiSOHxbSw37487XRd3.gb7wCTeGl2pgUS4HsSmiIoy V5Z5SWRPU4.E4byLRW_gx.CgG8w_gYCCZ1zcta.qbJeKTK9aeZiyXSEtQDcdUouASV3p8N1cxxCy D_e_hEkB8KnEM8uhcmN30fXGMbyLlbPLXTZmrmwF9qRRY88CITuhZtY0QO0FePzOQsIEf5v7QGFb z01NTXJXSJFhzTfi5xjDQT0XbGtZJuBlEvZ.mi.gh4VN4glyj7h7kVHdg4Pv36czQwTlL1qyJTY3 QwgGiq5U8MjKd4cikbEjSsEVK678N_DZIaejJRv97XfzpCGDd1drBwbCW9qmDz.jukDbRuO6a1O8 1v9jw8GOMRc2naif8VfpQdMJNTanruduFJoxyADNLFgAjhDpBvC9eo3XlRCoGFpbkVpB8aFtsAa8 g4_6lSXn05cM5GmjNb13IgRjcPCqcLntiaJ814i0Ber9Mp0tJ8M5dWdbjdgXcLOI9qsd8twokOzq M3MeEHZh9Hfstm2kI1coZWt6B2iiOVNyAOUbWwCuUDdXdlPMa6.EF8dhC3acE4nUaOZkpOrkO.e6 s X-Sonic-MF: X-Sonic-ID: 4180bfcf-a491-46fa-962e-27ad91874074 Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 17:50:17 +0000 Received: by hermes--production-bf1-865889d799-r6v2w (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 152c06392a2b898d44c06c6173da6628; Sun, 13 Aug 2023 17:50:16 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Mark Millard In-Reply-To: <3D94C0A3-0231-4C04-8C69-5516A4A41B03@FreeBSD.org> Date: Sun, 13 Aug 2023 10:50:03 -0700 Cc: Current FreeBSD , FreeBSD Toolchain , Mike Karels Content-Transfer-Encoding: quoted-printable Message-Id: References: <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE.ref@yahoo.com> <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE@yahoo.com> <3D94C0A3-0231-4C04-8C69-5516A4A41B03@FreeBSD.org> To: Juraj Lutter X-Mailer: Apple Mail (2.3731.700.6) X-Rspamd-Queue-Id: 4RP4n74F36z3Cc6 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Aug 13, 2023, at 10:10, Juraj Lutter wrote: > Mark, >=20 >> On 13 Aug 2023, at 19:04, Mark Millard wrote: >>=20 >> No arm64.aarch64/tmp/ SYSTEM_COMPILER or SYSTEM_LINKER bootstrap = compiler/linker were built. >>=20 >> =46rom what I can tell, this leads to: >>=20 >> cc -march=3Darmv7 -m32 -target armv7-unknown-freebsd14.0-gnueabihf = -DCOMPAT_LIBCOMPAT=3D\"32\" -DCOMPAT_libcompat=3D\"32\" -DCOMPAT_LIB32 = --sysroot=3D/usr/obj/usr/src/arm64.aarch64/tmp = -B/usr/obj/usr/src/arm64.aarch64/tmp/usr/bin = -B/usr/obj/usr/src/arm64.aarch64/tmp/usr/lib32 -O2 -pipe -fno-common = -fPIC -g -gz=3Dzlib -MD -MF.depend.libssp_nonshared.o = -MTlibssp_nonshared.o -std=3Dgnu99 -Wno-format-zero-length = -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k = -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes = -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch = -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts = -Wnested-externs -Wold-style-definition -Wno-pointer-sign -Wdate-time = -Wmissing-variable-declarations -Wthread-safety -Wno-empty-body = -Wno-string-plus-int -Wno-unused-const-variable = -Wno-error=3Dunused-but-set-parameter -Qunused-arguments -c = /usr/src/lib/libssp_nonshared/libssp_nonshared.c -o libssp_nonshared.o >>=20 >> using the pre-existing system compiler that, in your context, gets: >>=20 >> error: unable to create target: 'No available targets are compatible = with triple "armv7-unknown-freebsd14.0-gnueabihf"' >> 1 error generated. >>=20 >=20 > True, exactly the same error I=E2=80=99m getting. It was extracted from a copy of your log file. >> I doubt that "make package" was tested for being able to handle >> the lib32 addition to aarch64 builds for SYSTEM_COMPILER or >> SYSTEM_LINKER bootstrap cases. >>=20 >=20 > I=E2=80=99m getting it in buildworld, not in packages (it does not = come to packages target) >=20 Sorry. Looks like I misapplied one of your messages mentioning use of "make package" as specifying the context for the log file. Still, your log file does not show an arm64.aarch64/tmp/ time frame build of the SYSTEM_COMPILER or SYSTEM_LINKER bootstrap compiler/linker. There should have also been: tmp/obj-tools/lib/clang/libllvm...] tmp/obj-tools/lib/clang/libclang...] tmp/obj-tools/lib/clang/headers...] tmp/obj-tools/usr.bin/clang/clang...] tmp/obj-tools/usr.bin/clang/lld...] to go with: make[1]: "/usr/src/Makefile.inc1" line 340: SYSTEM_COMPILER: libclang = will be built for bootstrapping a cross-compiler. make[1]: "/usr/src/Makefile.inc1" line 345: SYSTEM_LINKER: libclang will = be built for bootstrapping a cross-linker. So it still looks to me like a build infrastructure problem: it did not do what it said it would do. I've no clue about why at this point. I'll note that clang and lld do not get per-target builds like gcc does. "bootstrap" means for getting version differences early on in this context, not the targeting. =E2=80=94 Juraj Lutter otis@FreeBSD.org =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 17:52:21 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP4qm6YChz4mSxC; Sun, 13 Aug 2023 17:52:36 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP4qm5jpnz3FZ3; Sun, 13 Aug 2023 17:52:36 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691949156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R7IsL2t3u5karf+KoNMWJsj0Q6QQwRfRx/Hykgu6L50=; b=UbTHbZsGdJeySzt9djaUR27W9ZmtbK6xiddVwrJYUwGd4IyLa+0xPpvmk4wSklBa6MynCa o/llaIirTtGeeEmzs43d505VAp5OmAwCmizFJeX4dFgJ3ILXELPYmu1j4M+FdsRka8oFak aqHPei6yld6bPm0OlJmUeg+/fAOfTravN0nMWmgJJb95Xxu3AEtj4oFnI3QxUZHlp2kt0B DaWp+zNc1aybFP9FhygGbdJTOs0EAHE92h2i00mIROIW5mO1MUYGNFFRYBErQ04KWegItJ Y5sqwCIyG+4i1M/0ZG1hyKkJlE13PQ4IUmooQfqTjJJTnGqn3YDBMW/egr0+7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691949156; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=R7IsL2t3u5karf+KoNMWJsj0Q6QQwRfRx/Hykgu6L50=; b=s3fRxqtz+eBAbFTBvTpj+8JBNq15NS0PtXDyecweinlVDsua6WLof9Sn5EBJ17pm3nTheP UDvKFce/PYR8LHrMUIS90HmvTOk+yO1bfJaVcYTYeziQW2qtkgWEQ0CA59c4fs7uXVR7s0 nAkEOZ0Sds4bIWInNZQsxAXzIF4iyFFtyuXrmocP37pwpQ3CfPztugcgSJ1iIROOSgEJod kiJZD4SYvzzi9Wo5ZMkJcQvx9MAx8VeevUMueIERdDg3nCLLBnJY8WdMQGTu0OFkuU7dNT nNwA11yXBCBMZBf5hPfFOR3u/rSwKapAXqarNsupEauFCuDf2oQFitfKPTrEtw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691949156; a=rsa-sha256; cv=none; b=YwtsP9ZFIyH9GEiDka9DLGpFSD0LewH0+w4s35u8dWNso1X0UszD8WJkwS879BwhWhqowE Eh0vniEquc7+uSjVqw0zzPkVpTvs75S0WEtxO8yzV90g02F86auSa+WZ524/PfN3UNzmfQ zf2uNmu56zz87YRww1qim1FTyJN+6KEQzMGcxJpxJELImFTYARwnpiPoKTHWVVs3icCHz6 McBe88hivQ0/vN5jkuxbN+Ve9eg0Vy1Xqgq7/7MAosxoyzVBlznaE9AAlFYe1C5igXiW2F OkqC7pxEUeWms41kNQIa5NlOfVOK41FeCbc/D4Ge0d7tHrvAaul5N3zCgMBY8g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP4qm3yxtz11H4; Sun, 13 Aug 2023 17:52:36 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [212.89.239.228]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id C0AD661F91; Sun, 13 Aug 2023 19:52:34 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: Date: Sun, 13 Aug 2023 19:52:21 +0200 Cc: Current FreeBSD , FreeBSD Toolchain , Mike Karels Content-Transfer-Encoding: quoted-printable Message-Id: References: <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE.ref@yahoo.com> <2A8F9F90-B03C-4A7B-9B03-B54386CEBBAE@yahoo.com> <3D94C0A3-0231-4C04-8C69-5516A4A41B03@FreeBSD.org> To: Mark Millard X-Mailer: Apple Mail (2.3731.700.6) > On 13 Aug 2023, at 19:50, Mark Millard wrote: >=20 > So it still looks to me like a build infrastructure problem: > it did not do what it said it would do. I've no clue about > why at this point. >=20 > I'll note that clang and lld do not get per-target builds like > gcc does. "bootstrap" means for getting version differences early > on in this context, not the targeting. See the other thread. After backing out = f1d5183124d3e18d410ded61e45adb9a23b23c83, cross tools started to build = again. otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 18:50:52 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP6744hMPz4q28V; Sun, 13 Aug 2023 18:50:56 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0.glenbarber.us", Issuer "Gandi Standard SSL CA 2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP6740GP5z3L1p; Sun, 13 Aug 2023 18:50:56 +0000 (UTC) (envelope-from gjb@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 2607:fc50:1:2300:1001:1001:1001:face is neither permitted nor denied by domain of gjb@freebsd.org) smtp.mailfrom=gjb@freebsd.org; dmarc=none Received: from smtpclient.apple (50.29.233.174.res-cmts.swb2.ptd.net [50.29.233.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 9E7A73698C; Sun, 13 Aug 2023 18:50:53 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail0.glenbarber.us 9E7A73698C Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: Lacking "arminess" (was Re: New FreeBSD snapshots available) (OFF TOPIC) From: Glen Barber In-Reply-To: Date: Sun, 13 Aug 2023 14:50:52 -0400 Cc: stable@freebsd.org, freebsd-current@freebsd.org Message-Id: References: To: George Mitchell X-Mailer: iPhone Mail (20G75) X-Spamd-Result: default: False [-2.58 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.979]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org,stable@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36236, ipnet:2607:fc50::/36, country:US]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; FREEFALL_USER(0.00)[gjb]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[freebsd]; RCPT_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RP6740GP5z3L1p Ship it! Glen Sent from my phone. Please excuse my brevity and/or typos. > On Aug 13, 2023, at 2:39 PM, George Mitchell wrot= e: >=20 > =EF=BB=BFOn 8/11/23 23:15, Glen Barber wrote: >> [...] >> It was discovered that due to human error (my own) the arm64 are, as >> Colin put it very politely, are "lacking 'arminess'". So, unfortunately >> the arm64 AMIs for 14.0-ALPHA1 are incorrectly uploaded as amd64. >> [...] > Many years ago, one member of M.I.T.'s Tech Model Railroad Club had the > annoying habit of assembling his model airplanes in the clubroom. Soon, > the club enacted a rule banning all objects capable of sustained flight > from the club for a period o ninety-nine years and one day. Thereafter, > it became established procedure for the club to issue reprimands by > declaring certain people "capable of sustained flight." >=20 > I now propose to express my displeasure with aspects of FreeBSD (which > almost never happens, honestly!) by declaring them to "lack arminess." >=20 > My thanks to Glen Barber and Colin for creating this terminology. > -- George >=20 From nobody Sun Aug 13 19:07:30 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP6VG6pTbz4q31q for ; Sun, 13 Aug 2023 19:07:34 +0000 (UTC) (envelope-from jrtc27@jrtc27.com) Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP6VG58d0z3NbC for ; Sun, 13 Aug 2023 19:07:34 +0000 (UTC) (envelope-from jrtc27@jrtc27.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-3fe5c0e587eso37009925e9.0 for ; Sun, 13 Aug 2023 12:07:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691953652; x=1692558452; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=R48JpoNpPvC5IGA22tRRiGn37CO7OLfIb4rgTnQSyQk=; b=kR0FzcS9UZzUKX5v/rrDYDNeX3iqPnWBvz2f86OQh1iMc5HyVcrSw/ia+7OEm6YgaT ftsREW7EyXJG4CDx8No1vdlgcK5IkByAt2wAXUGEPwOIhY6kmBSN1aK3dSmS2anr23Do d3b8c0vL1mJ3J4bDN1yKJY4Soh2y0LBc/7IlZDh74tgKVTRw4iA01Wir/yOwi/Q7Spgs eADukI2+q3PBktVrVQX0+QbSM7sFKOnwqnaRz6w0l7kNWGISlva0BN79TnHMYvdtWUoK ZOJ5oBRq6hwdQiPwwpdHZwTfgiqwDsGlygvHKie4lzzlZNW8cC5wF3Z0+kKTc2U2aBgy QPgA== X-Gm-Message-State: AOJu0Yw/kt2X4x18gIi1RfRtlkenZOwZFk9Qx5UdOYLqVvFab9Wgg0MW sbZpZiwxkTnYGaJiQSpwaD3yfxRsoeY+J5+PLBY= X-Google-Smtp-Source: AGHT+IEF24gUQtyeCnI7Y+6hVp/CUBValW8O08vxHP0Xr3SzRuadcvm6NkS1oK3f3Gxzj3c9+/sTtg== X-Received: by 2002:a05:600c:c4:b0:3fb:a100:2581 with SMTP id u4-20020a05600c00c400b003fba1002581mr6408021wmm.14.1691953652217; Sun, 13 Aug 2023 12:07:32 -0700 (PDT) Received: from Jessicas-MacBook-Pro.localdomain ([131.111.5.246]) by smtp.gmail.com with ESMTPSA id a19-20020a05600c225300b003fe1cdbc33dsm15041858wmm.9.2023.08.13.12.07.31 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 13 Aug 2023 12:07:31 -0700 (PDT) Received: by Jessicas-MacBook-Pro.localdomain (Postfix, from userid 501) id 03FDC6B76758; Sun, 13 Aug 2023 20:07:30 +0100 (BST) Date: Sun, 13 Aug 2023 20:07:30 +0100 From: Jessica Clarke To: Mark Millard Cc: freebsd-current@freebsd.org Subject: Re: /usr/sbin/etcupdate -D/usr/obj/DESTDIRs/main-CA7-chroot -s /usr/main-src -M TARGET_ARCH=armv7 ends up building dtc ! Message-ID: References: <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2.ref@yahoo.com> <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2@yahoo.com> <058C6DE2-BB5B-46CA-A9C0-D50293137A8D@yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <058C6DE2-BB5B-46CA-A9C0-D50293137A8D@yahoo.com> X-Rspamd-Queue-Id: 4RP6VG58d0z3NbC X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] On Sun, Aug 13, 2023 at 02:23:54AM -0700, Mark Millard wrote: > On Aug 13, 2023, at 01:06, Mina Gali=C4=87 wrote: >=20 > > from IRC, i have learned this is working as designed: > > etcupdate without -B does a build. >=20 > My interpretation of what "build" referred to for > etcupate was way off. I've added the -B's that I > was not using before. >=20 > Thanks for the note! >=20 > > Also, jrtc27 recently made it so bootstrapping, because otherwise BUILD= _WITH_STRICT_TMPPATH doesn't work: https://github.com/freebsd/freebsd-src/c= ommit/d81da4c98328d4ee3fe4c0a85f6874a3c69a1afd >=20 > Yep: that added what leads to the dtc build. >=20 > I still wonder if etcupdate builds more than it needs to > for its specific pruposes when it does do a build: > building dtc in order to complete an etcupdate just seems > time/resource wasting. It does, but teasing apart what's needed for buildworld/buildkernel and just buildetc is a bunch of work, and better to have something that works inefficiently than something that doesn't work. Patches more than welcome to make it more efficient. However, I will note that 99% of the time -B is what people should be using, so it's not all that important (it's rare that you want to produce an etcupdate tree from sources that haven't just been built). Jess > But, at least I'm using -B in the future. >=20 > > Kind regards,=20 > > Mina Gali=C4=87 > >=20 > >=20 > >=20 > >=20 > > -------- Original Message -------- > > On 13 Aug 2023, 08:32, Mark Millard < marklmi@yahoo.com> wrote: > > I'd noticed that the etcupdate part of my build procedure has been taki= ng much time. So I looked with a ps -axldww while it was going on. An examp= le was: `-- /bin/sh /usr/sbin/etcupdate -D/usr/obj/DESTDIRs/main-CA7-chroot= -s /usr/main-src -M TARGET_ARCH=3Darmv7 `-- make TARGET_ARCH=3Darmv7 -DNO_= FILEMON buildetc `-- make -m /usr/main-src/share/mk -f Makefile.inc1 TARGET= =3Darm TARGET_ARCH=3Darmv7 buildetc `-- make -f Makefile.inc1 _bootstrap-to= ols MK_CROSS_COMPILER=3Dno MK_TOOLCHAIN=3Dno . . . `-- /bin/sh -e -c echo "= =3D=3D=3D> usr.bin/dtc (obj,all,install)"; cd /usr/main-src/usr.bin/dtc;. .= . install `-- make DIRPRFX=3Dusr.bin/dtc/ all `-- c++ -O2 -pipe -fno-commo= n -DNDEBUG . . . -c /usr/main-src/usr.bin/dtc/fdt.cc -o fdt.o More may buil= d. This is just what was present when I happened to look. Seems an odd thin= g to be happening for an etcupdate . I'll note that aarch64 targeting aarch= 64 (self hosted build) also spends notable time and is likely doing similar= ly. =3D=3D=3D Mark Millard marklmi at yahoo.com=20 > >=20 >=20 >=20 > =3D=3D=3D > Mark Millard > marklmi at yahoo.com From nobody Sun Aug 13 19:13:20 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP6dH1g8Hz4q3gM for ; Sun, 13 Aug 2023 19:13:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP6dF61Qwz3QrB for ; Sun, 13 Aug 2023 19:13:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=EZfI7YQy; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691954016; bh=/p40Am/puiLMIxBeGX9KhTvIQcdGtdAorI0m5gW3hek=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=EZfI7YQykTZChTBy5c0g/5/Fz+Bsglh40vloscKl3QyiLoCqk+WIdOxL2QW5lwKW7UnlK28yd5xgVC3SVZcFbYpiE16VoBlWQaIuROHeM39DpDRA5dD9TfwrKNcMVwyPmuVjNyuIt3XLUp0IA6nZ1IYmXPv3CMYwyunDHjehVyF1N4ov47h9a4Svga0rL7RphDXAAgWhPXCrbKQbpE05E6EAGPTLu6unMstj43sM19+9KT65aWEA+onbEiFcOHqU0hM0oeoMbV1pQk6aIrSw+sm9k3bDy8H9jYYa/v4sZ77b1a/N/VuPqOOXzFODANUilygupU0SXGxAQz2SQSEhLg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691954016; bh=pZhsjhfWgwSyD5Gi1lRU+4UH+JSe94C8WWlHOj3zZNm=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=cZR4jnbdxfvzHOv7kyest0MZqNqcDKa6sk0DEwBdq5vMVC079BWD6y8y7TrgIj3O4JaSg7HYNmGvo8W3hf5o2XvBRssrC2RpT2lXFmzckdK6xumUN1VQHhn0P7weXDuHnI8iUkZ6YyJL1xzQIrrGZvqw4ySHMFLXNRJ+C//Ipd4jUEQMkeguafyHkTd5tjTuCFJ6oETqmylV6EQsWZ5n1y514sgc+tJmtKdVNsYXp7MFrx9rLotJ1Wbi5gmRxz3Zc2egZQmAcCexZ26s/BCzn0kC8I7XMq97UHZ5MmLGphOQTncJ+IZPNglk1hV4zEaVycDkM1UUnVvi4vGmQC5ohg== X-YMail-OSG: _Kggm7MVM1mRiCZyCvyn.noBIcTs4BuaEyFU26AvXraze4et_POLEw6Vqnnr2qG WPcR_6ad8c.hcDlCw8Ic_pwdsEZjDq0azo5zDCCodeoOMIDxAtP.ns9CM42ZnzBvHvzH_NZGy2AQ ZuxyaGa6E2OGJXf1dlHm.ewP5nd3cRg7k9Py6Yt98jCHb6u0oVqDSX_P.nVDWcnSJEHtO38KWsHo xbCZ1EcjaAxu7N1XU_N0Y3iVC0wGUjTVFGqHrY0IsxpNUrFDS5Jq5c1_tpBpek4xUZUhMIXgaj2y j9mq8DSHunN83H8QGXQRm8_jeomONQyX44TWtub.YPcAMh5M3BCPkWI_vYxXGs60FYBtclkUPZf1 Mx4RqvnpeEPYI.S.rHkKzS069lzf0GyJhz7g.J5_v5C0ybbJYS5z1KinBmMtdL3h2ZwGQFj2yVjZ u4crCJCF_vnNI2Z3JmAJ8JGre5GzZoS5TI953K0ccYkELLW42_.6614JNmf6.YsgHq5uZmIq39V6 1_Q2lOYSjvWISdeTpo9s5Pc3qlu8SZLdYvvCt3ciVIaMj3n7i9cdQnZNr7AOyiC57QtnrpK2A5hl JDVFKqOLYkdHDUtHgKIkJWTR5ExQVQROn2IzqD9Wtzntm2BxBGOnRAaM._jwJ.vGTwR1euqgM9qn 7O1sEm9aHkue86p8Tj5fWv2OOsvh6PqxFSgJMYoH9BXP1wPFtyjXQnD6svdWbSNx2LPhwDHK62DA b_8b8KZFH5iH6eL8_2hv2IO2vC25_4sl8aRp2cEg2ScvucndznQrGfPIizUlnEg4CR38c6WuRNVH po8xUE67a5JLr1UVTVCxqmFH8zo6Gil9JXKjg97V_KaeL8B5kaiXbgwmkYLLQSWlR6GssPhj.SDT KucgIXd9oUs1vukqCacv.6wmJW48W62Lcr6hNqIqjswAndcJMSG26MPmpM5TVhHc9fF5k70TMlyy hHcXzWdoqwR9yC5UrwXNuFdgTLo6y6dlFdR12KKp4wKQzvP4pZFm96Vlx4.r6I6GbxkaO4Qmvprh 2JZKkeGdd9xnmh3V7A5gR92aR5Ob7y5srohJmh9SKxuQnfrYrS30gxBnbSZAvjYX1qBLImPF927m EucyXluf51wUmKP7HWvgS9sO25ede2ejfK103KX9lhcwcP5wzleVmMztXFsLWZlmUN59sRJS7bSb J0wcCL3LoJTgNQ6aAXxsViCHhijLEe8yh6U3HAtqrBram_zfcCoWo90GwCxj4CAqh_eBwRcc1LNU w1Wc72Auz5eC0Yf0rvaoNEwpGLA7SJkCXtTYYIJlcSucJ.4BFR4FcoJT_DqsQW51hGOSpprfvYso XmTBoilwHIGz06awDEesd.Vpi_zSmtPX_qYLD3p_FvUMBkupwix55QEr1RCWfwZOcctEYWS4uR2u sxara1h5_ZxUVz9Q478hK43UhWe65I7vP_qAdF.C6zzpzgUN_30a4ikaQVNogPjDoXSfyNeOJ47N Z3ztRcASuIEcjrT52j_DgvsH0305Iv4fbI20kAu.bUI9eSWwD1Ac8ThnUM3ldOJvrCmwUsq5VRCE hWd5fZPj4uPOsaSgIlin.cFHohNFD3iJQh6miqBk8.KAmjN7ePgQnasGpxSF3.zyVeXsQpox.7YL YkTzmx4m1JBUr0_aJoRqC4KP8P3ht3pFuF9qsNSd.SCksdw5KD1baTwvreRnBv65eGlKtEOqAfcf hKJKI.XBhHxuZhxk1X9jN9i7KBzlvQY9dIwX.2BrlYSPKmpBi6RNFZECHXT66CkfRY8dQNorihCh taJtPsZuiuq.ZbqLAecHY989UErMIb5._o0jz5M5kZ3kITuIcdHHGjFL16_kW4t6LIT0.xZb6N4K 3X10ybvQmYKgDiTrsHOVh7pQ9y7H_3Q_.YSrPozNpx37HPP9EtI3A2maRfY4ya5lBaDti.0Pt8Ex 2qTAyQXZdYEt1Xfj8ng7odzUY4lLLucuTKbjPxwSx.Kj4EEH2sdMQiUR9LHnTYkkQzEZN5cG8cpR jFcNqyUVO6DqINBqwbC2TAqXKpJ3eZ5OYpw88KJXyhbQ0mBazW.rM28f5U7eFOiU8fRxF1gNtQPE paMt.wTe9V.a_q.x7YlMPCcXCXBH8q0vrdMnRMp3PetrBo5_nKVO07xqUcJfd4.dGliuA7JymYoF 2uP7JWjmGcbTqpofwnuFtCQdgIXUp22gSoJGIMaTAgukifku6ZEGAtmGpYn_m4A9lOVphRrWSJQ5 TCwRsY1soJOxOSoRgsMMOj4_xhAZ.IlVFlEYUpE3jpjXpGFNVaSUKOcN2Q9.MHzj__D8MLvNTBF_ f X-Sonic-MF: X-Sonic-ID: feea1998-ff2a-4660-9e0f-68b4240eaaac Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 19:13:36 +0000 Received: by hermes--production-bf1-865889d799-7x4p2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 78c49f9fe505a6f681bb0438893ea1a3; Sun, 13 Aug 2023 19:13:33 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 Message-Id: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> Date: Sun, 13 Aug 2023 12:13:20 -0700 To: Juraj Lutter , Mike Karels , FreeBSD Toolchain , Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RP6dF61Qwz3QrB Juraj Lutter wrote on Date: Sun, 13 Aug 2023 17:49:16 UTC : > > On 13 Aug 2023, at 17:22, Juraj Lutter wrote: > >=20 > >=20 > >=20 > >> On 13 Aug 2023, at 17:17, Mike Karels wrote: > >>=20 > >>=20 > >> On 13 Aug 2023, at 10:00, Juraj Lutter wrote: > >>=20 > >>>> On 13 Aug 2023, at 16:55, Mike Karels wrote: > >>>>=20 > >>>>=20 > >>>> lib32 is not built until stage 4.3.1, after build and cross = tools. I tested > >>>> a build just now on amd64 with empty /usr/obj, and it worked = (make -j$NCU > >>>> buildworld TARGET=3Darm64 TARGET_ARCH=3Daarch64). However, the = host was > >>>> approximately at the level of ALPHA1, and the first lines of the = output > >>>> showed that a cross-compiler was not needed. However, my earlier = builds > >>>> that required a cross-compiler worked fine. > >>>>=20 > >>>=20 > >>> Earlier builds worked for me as well. I use it for =E2=80=9Cmake = packages=E2=80=9D that I use to upgrade my RPi. > >>>=20 > >>> It definitely worked on 20230712 when I did the build last time. > >>=20 > >> That was before the lib32 addition went in (20230725). > >=20 > > Yeah, that=E2=80=99s why I will try with commit just before = eafd028327cee688b54bc526e088c2a3b98f94e0 or > > with eafd028327cee688b54bc526e088c2a3b98f94e0 backed out to see if = it will help. >=20 > To me it seems that f1d5183124d3e18d410ded61e45adb9a23b23c83 is the = commit that broke it. > I did =E2=80=9Cgit revert=E2=80=9D on that single commit, these were = rebuilt: >=20 > Building = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libllvmminimal/Supp= ort/Path.o > Building = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libllvmminimal/Supp= ort/Signals.o > Building = /usr/obj/usr/src/arm64.aarch64/tmp/obj-tools/lib/clang/libllvmminimal/libl= lvmminimal.a >=20 > And cross tools started to build as well. But the offical CI builds on amd64 do not have the problem. We still have not found what is different about your context from the standard context used for CI builds and snapshot builds, all of which have worked find building on amd64 to target aarch64 with armv7 support. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 19:26:59 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP6wq6ptkz4q4Rm; Sun, 13 Aug 2023 19:27:07 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.zefox.com", Issuer "www.zefox.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP6wq3NZZz3SmQ; Sun, 13 Aug 2023 19:27:07 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Authentication-Results: mx1.freebsd.org; none Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.17.1/8.15.2) with ESMTPS id 37DJR0Ng025403 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 13 Aug 2023 12:27:00 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.17.1/8.15.2/Submit) id 37DJQxFr025402; Sun, 13 Aug 2023 12:26:59 -0700 (PDT) (envelope-from fbsd) Date: Sun, 13 Aug 2023 12:26:59 -0700 From: bob prohaska To: Mark Millard Cc: freebsd-arm@freebsd.org, freebsd-current@freebsd.org Subject: Re: alpha-1 armv7 git failed: fatal: pack is corrupted (SHA1 mismatch) Message-ID: References: <8DCC324C-966C-45DF-88F6-A442E348D4AA@yahoo.com> <2F080FD7-B63E-4FB6-A165-E24A3DF2730B@yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2F080FD7-B63E-4FB6-A165-E24A3DF2730B@yahoo.com> X-Rspamd-Queue-Id: 4RP6wq3NZZz3SmQ X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US] On Sat, Aug 12, 2023 at 08:45:54PM -0700, Mark Millard wrote: > > You might need to use the ssh alternative if your > context allows it: > > ssh://anongit@git.FreeBSD.org/src.git > > There was a time when git fetch proved unreliable > in my context and I got around it via ssh use. It > also took less time transferring. > Seemingly no dice: # git clone -o freebsd ssh://anongit@git.FreeBSD.org/src.git /usr/src Cloning into '/usr/src'... remote: Enumerating objects: 4323641, done. remote: Counting objects: 100% (381285/381285), done. remote: Compressing objects: 100% (28204/28204), done. remote: Total 4323641 (delta 375529), reused 353081 (delta 353081), pack-reused 3942356 Receiving objects: 100% (4323641/4323641), 1.54 GiB | 656.00 KiB/s, done. fatal: pack is corrupted (SHA1 mismatch) fatal: fetch-pack: invalid index-pack output # I've added freebsd-current to the cc list 8-( Thanks for writing! bob prohaska From nobody Sun Aug 13 19:36:38 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP7861YHZz4q5DK for ; Sun, 13 Aug 2023 19:36:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP7855rTPz3Vlq for ; Sun, 13 Aug 2023 19:36:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691955412; bh=rzTRHyCxVc2v/h1OY8Ze8e6NuQZlAsGBQbwL+6MbzWM=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=JElk+XQtBmM2H4h88XqNTfBQADwWD36ctwd50bY/rE3ommiO6o+oDV/5pcsEAzum7metbjkJvHi3JNluN6Sm/W2gi/Q9KnT/vSOR7GxtCxz0J0M4K55qcweEnSQ84mlkwOfND9X02kwbb8LouD71LQli4P6ZulSQmmIw8cprw7LAo/s6J2KgXyquTxMAfbaWDlfYLVCWbJq62MtYpVk2jUQDuXAV/Djy+9WdbHco124X4GSpnk1UyZFdRGbSdgMYnIw7QmQ5Mxwta1JmlHY2iOYSuBHH1xk01X31I+v5dtyYJaankn7FVZbgTqgt8+xThsSYh/ABb6KENbQmtnXaqw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691955412; bh=XUZ2SRX/y7M4/DffQw/QFcaLWWA1SeE9NxP8pGDy/Kr=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=blqTGkRxAaaS2liZdjVhcGmdD7bYy0MbK04NR8UfS5TbJgJV67v+OMijSH7e3Cj7ty0/o2GjRNIHHdJFu6Qwta3yMRukqTxv4csiPG367JZLeBOQzV032n6i2cOwTeWV6vfN2xM3Aivh+h3zJEsVXWb26tm4EpkXy/aCX9+BEeuya4HUIVstY8qInqSA38rE1YSARQTsce2Wui4L2BB0WKTcsrzhOC01uM0iKz9dSQ8Z9aSHHZYFfIk0ukfrymS9tePcykZR39NpCJ60VLkxaW5NkCgmvnNoWFw0ce1Ww5qOx0ueGlyl2nTtqoq+TgheJkHP0OaG+c4x47G0eF6IAg== X-YMail-OSG: xRFqGGIVM1nMchHXtkBcSmc_QWlfP405agqdhKFV.Btic6UtIF87IQJJffaZwUM ym1qPKgVTRfzpgPVMuoF2mbOkgf5lmik5jW28dtz9K83oZUDslAXYh_klbxmFsbt2zHfq9ohaX5w y3fs4uDJcYMmQpcB6UgIdetZfuxonqKj6IsTdMolw165ifjvhb21N.D1exxQa0k6z86Kjci1nb4_ qCQo2fNJnAtWPNZhlj1YZVO_1nBSQc6xJbJcaxwTyOvzhgRdGSMe_qi9Pjphs_2mcqMHsMMxqQfC 6CWBDTXiRdyjX1717kCbJ_2uBllCaIF7sTmef7CwHOa.528E9IUzH5vzChEcZ31KvjJAaPUXlSy_ T._nlRHoWX20Y9Ee.2b.ckjRgpmh9boZ1ufwant17.7JtmD0RPtjicBLEauhD2w62RHlK6Aybbfz HL8f0P8ih_dlbdvEgt8vAIzJPLPFppYtFEA83eCtOfkKoDP.OxmL69b1RTECQfhr2g7xx05WQDAZ JzG1RGtH5iKy4_D9pPA8go2u6jadxLxzifqwxbzkjEEC5seUPgy91K4MuFSMiP.lS.811nirdr4S 5B9xR3PFx0q_nDKIdoxlibRWMTkaDdNw2z1rIWi7JLuSSH6pQhpv2zRFQCkD16j00Q98wudwJbfy OVgxaXnyUWfG2J2BhK2Kwm8O6pce9ylsRVlOmEF5.2ZPHv3Z1u.sZoQU_tQ8bwh6W99DMnuu5lpq I.HCdFP14_3iqNVgBMN1X_QQiGdzySgHEQDmOjP72dH7aKlrzcEXNR2h6Y6fOEnJutzpOcz2YlXr COWajsIt_nAcDnJG8u6DqhoAjTfKdwBV_3nyk1cZvqUPYJnk.vy1i61A2LDgxV1guW6wGDOxStLe a56MG.eipkSekAuV0o.66nsFqKoW3w2DKLP4oNtEX.hgahzcABg4TOr6J_Ux7D2X_NHHeMLSjjn3 SbM0n2pnm0A3.xidSLnzxsUlPTa.qgTkcLZDnga17XQukqtns_Ordjki6qcAjyHplbHb1cmfSInj aWkj4LBDXzEdLeLqgVnURUHMInEIgitGtiaizCZ9we6fICY2umbymRuylEcjerMQNkrITEd6xMst jVyJldyC1iZ7kPSPEp.MBv4N.GDEKSuhc4vvnura9g58h9Bgigca5.K_T6vy1m7KSve2kQTUZm5Y 1AXKgJpCf9J_lXV_ZfnYUQFqiDTMjmLBsuxQzz.eLwwCh.vFmxShhPDSeCnl.KXQbMeyYPG7TVF4 MSc9SWkdPh9ZlJVqdP.KzTV5K3IgPKFaPyVYIFjpSJob0MhTAKKz1JNnHzp0ItAAQ3UtZxp3OcTF R9.VrNf6eBcxtZrmxDJW.mfg5P3JjT.eJqSpYjv8j8TEIZ08YAQwjRJqSV8rwzIhS8QN7zwOUnlA cwzCGj233CqlH9nxx7.XLKTiJF_RsDqkmwr.WGNLReHhO0A.lK5zi09j8N_.36f9EW7LwqjWyEmn yAfFWSYyOdiFpJCeLWxaEOaHk.9nlCRWoHkyGvjBAAn2IEoXDK0YPU3h68wytPk_A4Atpv26MWYp H.rdzOsc1NsW4SI7SOa8T1q_7nX94RF8wcDjxtOQR3Mxy7suVe6YJTSTEnvTx6JdVKbrQmBnNwg6 fKm84wF.uEgYKPBfanRMF38Epm9mGXO.yxO5vt5wzKZyhUdLZXgwYniBDnydFhToZKAe1h0grXHd 2wn2sO8jCLAN_aO7pJ_tsbOqrfFZ58E5owWG2K58zlWNP6PyH_x5ujRrtuBu_b7lW.4jJOrCKAIR DU6.ivD8snhpj6r2Woq9tcfOREwAC3CjdPXq5Cjq_x2fKXzwk2LfNrvjn62GQuxrMj7xTtyRs31O QPKsCYtLC9PRO3s_gKdUAy78udRQIj5Iswg9JD9HmmvVS1KAmmhon3Pluybb_WoRMZpwm0uG.JRe 4m2g5ieSJCUY5Ad7MQL5Tbw7GlFZUd6VnYhTVeKMChNYCl.6n8PuS.C1il.gE32869Vfi_Jyjnau MOE0R9_rbqvbCYq9XGJEZq6TIfT6ipRz.3U_nYr6kqZRttiySrqzlCZFqDKhmQ3DIUPmN8U2bJke r_EwVyLqd9pE6RzNtnB1p3J0bzwGjBGwAvmgPWtrLk9yfudeewBNQM7BZfAB5Xl.4gQucpYXsdH6 .SxQnWFblnTPTThDooDe.T2g9LV4.XtaFoTX4bH0rv7WnVO3BKXDXj9shZjXIBP9UQZ4getOC32T 8hysEDA1Dv.1SS9zJoBoEfpxDZZjBis07zDuarVM.ZdxHgCkvKWPnfJBpmnoC75xLgsSlpr4Xp92 vOQ-- X-Sonic-MF: X-Sonic-ID: 155fc084-9c04-4623-abbf-46b66150a0dc Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 19:36:52 +0000 Received: by hermes--production-ne1-7b767b77cc-fbxjt (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID d6c1d21b18bd012ea5f0370053163956; Sun, 13 Aug 2023 19:36:50 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: /usr/sbin/etcupdate -D/usr/obj/DESTDIRs/main-CA7-chroot -s /usr/main-src -M TARGET_ARCH=armv7 ends up building dtc ! From: Mark Millard In-Reply-To: Date: Sun, 13 Aug 2023 12:36:38 -0700 Cc: freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2.ref@yahoo.com> <3EF7CF20-BFC6-4A25-8806-633CB2A3C4D2@yahoo.com> <058C6DE2-BB5B-46CA-A9C0-D50293137A8D@yahoo.com> To: Jessica Clarke X-Mailer: Apple Mail (2.3731.700.6) X-Rspamd-Queue-Id: 4RP7855rTPz3Vlq X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Aug 13, 2023, at 12:07, Jessica Clarke wrote: > On Sun, Aug 13, 2023 at 02:23:54AM -0700, Mark Millard wrote: >> On Aug 13, 2023, at 01:06, Mina Gali=C4=87 wrote: >>=20 >>> from IRC, i have learned this is working as designed: >>> etcupdate without -B does a build. >>=20 >> My interpretation of what "build" referred to for >> etcupate was way off. I've added the -B's that I >> was not using before. >>=20 >> Thanks for the note! >>=20 >>> Also, jrtc27 recently made it so bootstrapping, because otherwise = BUILD_WITH_STRICT_TMPPATH doesn't work: = https://github.com/freebsd/freebsd-src/commit/d81da4c98328d4ee3fe4c0a85f68= 74a3c69a1afd >>=20 >> Yep: that added what leads to the dtc build. >>=20 >> I still wonder if etcupdate builds more than it needs to >> for its specific pruposes when it does do a build: >> building dtc in order to complete an etcupdate just seems >> time/resource wasting. >=20 > It does, but teasing apart what's needed for buildworld/buildkernel = and > just buildetc is a bunch of work, and better to have something that > works inefficiently than something that doesn't work. Patches more = than > welcome to make it more efficient. However, I will note that 99% of = the > time -B is what people should be using, so it's not all that important > (it's rare that you want to produce an etcupdate tree from sources = that > haven't just been built). Thanks for the note. -B is not the default for etcupdate. Because of that, it would probably help if the various places that document using etcupdate for the 99% type of context had an explicit "normally use -B despite it not being a default" and "omit -B only when the source tree version has not been built yet". Such might be easier to follow than just the existing wording. Although, the lack of -B for diff, resolve, revert, and status is messy. If those silently ignored when -B , the instructions for use of -B could be simpler. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 19:45:12 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP7L10Hnkz4q5cV for ; Sun, 13 Aug 2023 19:45:29 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP7L06FNFz3WqV for ; Sun, 13 Aug 2023 19:45:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691955926; bh=cVf/QYr6MqgeSnv2QEaSdtrRXXTmjWrPcmZ3Ja4TFqk=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=cSfSL53y6jtPF3qE/+I3Nq3VulJIKQVY/pQFV/C7MOJcgU8llZxMXRtezE9Gn9WlZZaUPAAE2zpE3vG03Hy6sJ+fszMVfs1m3ZGn6JRyR0U+ArH5c+cZaDkomiuBcC15LC5n3zEOuL+BfU7Vrx6EXwWj66B6JyT7ZKrj/OzKPGMtc1qdzSquMYOmPQtq8K+usOWoFZLatDzz2dvCUMbZSSqBB8QdsDnMEVdOWt0bdq9tadcvfoEWkwRj8wv2P+WYBd7T1rNcrJoG0zGG3TxvbcNuWlby3T8NFCo2/O1rKM5g/dOa8U5qdXNCZwA+u3ZrfHFK2ixjGljV6TsMGamZYQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691955926; bh=oWstmeXeFshIJL6GHNXs2pfih1nbGvl85/wc786n1ne=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ly2Gy+AHw25ikrqAJ2t9rdmwi60By5OQO2XMiJLRgtd3JbXiWeJVmv4ehwLGDdgpCtVlPAeiXSIqSRmTVYiieoc2CmY/GqQm54iDeQxd+mxZbl01/Cm4spyPEatssT9u6nrwEPweZLsDSGPK5efaUaLQnzQkPhfm/RGhVYSIkNkBxOatURs2fkhSqIklZ68yAX14ZiDuf5diJGUAaJ8aCTLp+b75A4YSU7j4Fk0dzH+/xiBYejk4m+XtPBwHno642rGO+1PP9tG7XmvhI6+gXK6x7YH9IC/ieTz4ruY33m9rLFP85sDeyIxz6ikL2TbWPbIib6lC32CBhH6kXNUsGA== X-YMail-OSG: VvoDbpQVM1kYiHcQHUTKeRQdauz9GWWqHZA1NWGxnt8zQeUvq7ZRySZ0WWM7Pcd 9i3UjYVR5P17lfJHUBNRiKcXw5xqcpyixI9UZyETcjn9sizVZ7q.qhwnnQ_5ZAkiKhOBP5wzp03L KF9qOgPOswDtZnWJEHxlyL5IVVyGWKzPl0yX1B0Wkas9aewTFa8R2imSRRk2MH1icIxIrAnnASCd OufgDyvzrur8XUPJrXUP.wq0iku1uXprkFOl6y3HITI.Sq0CWIyKzyK77V_tRwJyhTwLbD_7AZSQ VkCDrjTNeYjh8wZQFFY8i2RRypGQJNKXXBPZdTntcXSTuSvBShxonMykoxSxFLOWs3E6iWjEvB_l vZFeU9EtbWbMviwT273tQlb2Xn7pzCn7VAcBNWeB292_OruYMf9RNsaLvMCRTlo4l0pTOomdUMOJ Uch5m73mxFtfWu5nIqz87LIY8gePTD9U3y064TdxEFOKoJn9nHTaIwNPiiJuXlHZjIsH59lv9aqZ 8mbtu9ewtZ83Nx3.LOip_G_dj9Akoe8JyIUPM4OSTnBtAmjOpL7eyFJ27FHTC8Ns_1w3tgK53E3G pGJjvUfZaxxV_rGWEQr1Zpja0yIEwQYGI.jYc8tsSc8qx1AEuLTVs7R7aEdZ4yFeOFEgGCGRoFj8 Uwenp8NM8zi4dubkRXT7YYbuzxr7NmdAB6Ox3P8YGStVUimxk9GkKIvlrbrAw3CIPWKDyF13F9BG S3SpUPPHkxHIr6Cpzsl3bqymkbKF90KAA1ccaxs8MUMW0n9oLtMtjniSFtoYcXPz8f9KrOP1gTuG 9qI1.4SQJjJJqSCLgNKA46Bm8_nEZ_5mu39EZZvpFSDcUL_KjKYurZ2aI7OnmjSHPiaXSpxRIh7O 3QnF3se2zA6TmYZJ8rbCiGSbIE41vvd6eyO4NlEsFr5G.AFvaFPauYXgFWHFv45vFxfaKjB47ias 6n4WM5xHeJgckE8v1sFtGomRv8y96oTthJMBhFJx1Qt18MScydkFekKJ6lrwgtY4KveS0ZV6dIBn WsrniSQEF5CWrr4K2niDzwH5pKUTO0ayZl7xZg_VKHLur.BSK7Jwf9rDEXlA.eChoJ0F8BQlvKEI etj86LfarqskUdQLWe01bZ0DUa2OKdQC9dgEobG.bw0QcTx_u_T0GvEn3W3EEVRaFLDlRech4MZJ tq.Ck9KhqCFwOhbqVlUi_7rezEeQk9L6vHYxdhJqEj_jQ7eKLw17N7WamDzYymCLY_kZcAA5toAg n0.cmqpcDvy5cRd6T73awhjtS3vi115rGM1tSad5pbvrOaBToVJmoAPttpUXzycsNQyyOBPAAt.O AN9KH7L.1e1AmZUVLDt442EQ.39GVruwX6H_E6DfbZ_39MtopX6fPsYpolMcub.84qChpsa4mSeW yeWs_f_5irSbaLRVq1hxrbaeppS2ivuaNOPiWRTQ_.asD0z.EE12x5.EeFEeilgDKMMvfPwdTDXo mfajyieYQpB6UtOwxe3A68gIGL8MVkbHihM_wzRAu0VfB7LqsulNoIqv_h97qx8JYWlCK9Gfe1t. 8kR23wpatvy2JbAKMh1BwsyTkfFXtyYPPmRbxSmDkW5Iq6dTIkZtBprD4V4ljf1sBlee71uWoA53 t7Ss0ATJPhSEuDmrbY.yhlBLXJiWSjJpDdESzUObrM6Uc.WnJ6vhQ1DMQ2iuL6pEcimmItYJMk2k yI.yv9loMwznZL4jJEe2PbPjdRznDsR60eqp5muioT2iGHAfd07ewnDsh5OEbFNdcLIE2y63l1Ki x3HbLIRkEDxton7YKpMjniSmReJQZ1dhvVrXocD_DezQLjiQuRl5qgQLxVgmxFw2.GidlxiTp1jD 18f.fQvV1hL.HyTqHrUICyJLXs2pMA1nvf80x3bKpYwvD5kJJNFfPzWtMYWKolFLoeS_q6ZGLmle 1ggyVzZgehVZJNyvrRFD9rAHfo9RC141NTg5dsnShbm3gQnrqkGFM6tYbvWmU_AaNTHKa3Mdjxx5 4WEOlj9iGVRYli1BbA8QMo_euPpxs2.NTggypt59THkLVsCT3M_vuF3CBK8aNMVuZE_sixvyl3DQ BXEtz44tAEFrO_cMcleVGVigFz9Rs4lR8z9xVQMJrReae9KZ8zBJSMY73UBPbOK5GpvCjdIDRGwn 84RxWZcHsQDRKmrjh2oX3s8qynC_9AAC_9GuSFaKTTMVdd0RJQkZzfVxmKsy0e21rb7TD2c37JvT WWJC6xVvzcgdXFJIrbziWn8aMX46OCoTli2OFYmj_SFaFEzYKUcQCfrUXePHIXLHBo6JTbgSMDso Ark4- X-Sonic-MF: X-Sonic-ID: 6c9d9a70-2523-4b50-9b9e-ae08b647e128 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 19:45:26 +0000 Received: by hermes--production-gq1-6b7c87dcf5-792fm (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 32281c6c0b5f075daccc48d059ca61f9; Sun, 13 Aug 2023 19:45:22 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: alpha-1 armv7 git failed: fatal: pack is corrupted (SHA1 mismatch) From: Mark Millard In-Reply-To: Date: Sun, 13 Aug 2023 12:45:12 -0700 Cc: freebsd-arm , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <8DCC324C-966C-45DF-88F6-A442E348D4AA@yahoo.com> <2F080FD7-B63E-4FB6-A165-E24A3DF2730B@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3731.700.6) X-Rspamd-Queue-Id: 4RP7L06FNFz3WqV X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Aug 13, 2023, at 12:26, bob prohaska wrote: > On Sat, Aug 12, 2023 at 08:45:54PM -0700, Mark Millard wrote: >>=20 >> You might need to use the ssh alternative if your >> context allows it: >>=20 >> ssh://anongit@git.FreeBSD.org/src.git >>=20 >> There was a time when git fetch proved unreliable >> in my context and I got around it via ssh use. It >> also took less time transferring. >>=20 > Seemingly no dice: >=20 > # git clone -o freebsd ssh://anongit@git.FreeBSD.org/src.git = /usr/src > Cloning into '/usr/src'... > remote: Enumerating objects: 4323641, done. > remote: Counting objects: 100% (381285/381285), done. > remote: Compressing objects: 100% (28204/28204), done. > remote: Total 4323641 (delta 375529), reused 353081 (delta 353081), = pack-reused 3942356 > Receiving objects: 100% (4323641/4323641), 1.54 GiB | 656.00 KiB/s, = done. > fatal: pack is corrupted (SHA1 mismatch) > fatal: fetch-pack: invalid index-pack output > # >=20 > I've added freebsd-current to the cc list 8-( >=20 Wow. I'm going to suggest doing a clone (to a temporary place) on one or more different types of system, such as aarch64 or amd64. If, say, aarch64 works but armv7 does not, then the corruption may well be in some armv7 FreeBSD handling of data transfers, in places common to both https:// and ssh:// use. Note that, if you get a good clone, you can locally copy the tree over to the armv7 media. But that is not the point of my suggestion above. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 20:19:05 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP8525v5lz4q7nb; Sun, 13 Aug 2023 20:19:18 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP8524w95z3bTK; Sun, 13 Aug 2023 20:19:18 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691957958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dRZtPBZps98hs07NQFK5me1loCuBSzMyvsKZwJI32Og=; b=KqmtCC/QOXrqcURdM+E1ptwFAClzOlTXAlpbhekdVeMpaSWVC6yhnTUoJgMBFOvEpTovKg jwSfvvS1C0+GadKA+o/hZHoAcU7mWIoAr4/3cxZnSCfnQFa8SHg9e52OUOXI5YUAB7S5B+ ERReMV6EXWp7nU2woMHhw1D1mVFan2zl+/qdP8XxGn6RtvoL6RKDiILYFdZFhN/ifIE1L3 0q5aQ7DkFonmUcQGRmFhMcVT1ge+6/UzQHnYsrjxdd/mWl8JjkoOVMPI1BvQVN0n6OaumQ hqCLnY07lwhkBTQQjSuCjHFuT/bKR4zq0XYjFpIf9UwBeIEYl3epnY0yJA67zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691957958; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dRZtPBZps98hs07NQFK5me1loCuBSzMyvsKZwJI32Og=; b=f7CoRO998RNoyX3/97PQ7rNNPhMfQug76a30fN1ZTYtOtENhxV+L+cEFDLo4yJ67r6HBNq YE2kgsniU17zs78gbJY6niRsExNgdqBUVDtZW+GNGNs/YjfYZUrKunzqtxbSHVO85SnmEb UPI7VszYPDFVb4kjUFadj1QCJOuhnEkr5/8N363tnztqtHsdRle3OXByRiaRKTOYMMPjia MtQmxc4lwzSSABuVBhf5F93zDrgeEqDrOSGhcHu1y+1+zxjNSqFjnioZdePLll+wV4mhnB 4oP9WeCbX33Wb2E896uTny78pPzxXir3tAeFwN9grfBhOByCgdwZucarGNnJdA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691957958; a=rsa-sha256; cv=none; b=O5YwPs85LfHq5w9/14uZE1v1tVwcCyhKvSwiO0Q5k6n44+frznnbzDR14siEs9IRD4shYU Xao7JO4010DGbGNFTiET3UgsqFcoTwtbmjlX5BFy3Ww2Lt/Xrrn1A3zcuMAPuYhv5qywA1 Pw3qCvH0jfgMwagHgoO4DuaOKmQ6EKkF3YdR7sHlAaDqLpO6PF7TD2zG/L2FwnP/JjX5A8 QW0LmP+hSaFAUyUm5oxLaDomqal+nvnEkYLRwk4sZdVxNDL67/i/dkRH2hABvea6COjokk f7D+IvfPjji4h1249f0EcJdSQYhdBI7CRGyWo5wqkBXFG9gTBr1Bb2Jl+YjoGw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP85239CKz12Gk; Sun, 13 Aug 2023 20:19:18 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 9876B61F91; Sun, 13 Aug 2023 22:19:16 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> Date: Sun, 13 Aug 2023 22:19:05 +0200 Cc: Mike Karels , FreeBSD Toolchain , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> References: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925.ref@yahoo.com> <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> To: Mark Millard X-Mailer: Apple Mail (2.3731.700.6) > On 13 Aug 2023, at 21:13, Mark Millard wrote: >=20 > But the offical CI builds on amd64 do not have the problem. >=20 > We still have not found what is different about your context from the > standard context used for CI builds and snapshot builds, all of which > have worked find building on amd64 to target aarch64 with armv7 = support. My src.conf: WITHOUT_PROFILE=3Dyes WITHOUT_TESTS=3Dyes WITHOUT_STATIC=3Dyes WITHOUT_LLVM_TARGET_ALL=3Dyes .if "${TARGET}" =3D=3D "arm64" && "${TARGET_ARCH}" =3D=3D "aarch64" KERNCONF?=3DGENERIC-MMCCAM REPODIR?=3D/data/poudriere/packages/pkgbase MODULES_EXTRA=3D"rpi_ft5406" WITH_LLVM_TARGET_AARCH64=3Dyes WITH_LLVM_TARGET_ARM=3Dyes .else KERNCONF?=3DGENERIC-NODEBUG REPODIR?=3D/data/poudriere/packages/pkgbase WITH_LLVM_TARGET_AARCH64=3Dyes WITH_LLVM_TARGET_X86=3Dyes .endif make.conf is empty, git workdir has `main=E2=80=99 checked out, without = any modifications (git status reports clean workdir) otis =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 20:30:48 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP8Ls3GsMz4q8WG for ; Sun, 13 Aug 2023 20:31:17 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: from mail-vk1-f173.google.com (mail-vk1-f173.google.com [209.85.221.173]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP8Lr48jHz3cqV for ; Sun, 13 Aug 2023 20:31:16 +0000 (UTC) (envelope-from 6yearold@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of 6yearold@gmail.com designates 209.85.221.173 as permitted sender) smtp.mailfrom=6yearold@gmail.com; dmarc=none Received: by mail-vk1-f173.google.com with SMTP id 71dfb90a1353d-487db6c9059so280855e0c.3 for ; Sun, 13 Aug 2023 13:31:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691958675; x=1692563475; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=snpeNgpUtgp2AVdWRaF0NEM74z+ypp2upDRmDTmjL9g=; b=Lbl/uOh8psp6pmNork43/TNIRJyGbKkU0AHLSh2fk4S0I3OvgLabUGT0U42Al46YqG Rhx4oEW5WuDwaRI9AZjVlrCp0b7drHEQTs8oOeQk5ss0YMN6Cf0OE5Plhcw5PG8SlOiH GfZrHCZfO0bRZ2e2oHYBEUZtKaU0hMpqkjPfnkLYoS/mIY0kfUCVJcNxx0MP1fkI8+gn UzF1WmCCRhNjRR+e0Z07kPelMiG3g0lLUBNCtU+y05I6dlhxnDV8SFUkGK1T+OAPj3FL MBBh2AhKqwAcHqX+aDIIm6PrLt+zW/cxlYg57Ok/cDIHqfg8qCzp4OHVNAk1aPFxu4S0 G6lA== X-Gm-Message-State: AOJu0YwdekJc6Jkz9NfcwGdTFjsdCrNdu35VVwKh59aFI7Jsh1/T3rAB OC8cLboBtVcwkPS6XQ74ALUV3mrBxPUDDg== X-Google-Smtp-Source: AGHT+IGAB43/wJLWRvFvCHB4auVtgYKBFZwqtQG8+oEr5RFpqWbRRImD7eYP9usZ3UYkIXP1ky6Fow== X-Received: by 2002:a1f:bd0a:0:b0:485:e984:779b with SMTP id n10-20020a1fbd0a000000b00485e984779bmr6421600vkf.16.1691958674975; Sun, 13 Aug 2023 13:31:14 -0700 (PDT) Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com. [209.85.221.175]) by smtp.gmail.com with ESMTPSA id d2-20020ac5c7a2000000b004873452809fsm1326623vkn.42.2023.08.13.13.31.14 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 13 Aug 2023 13:31:14 -0700 (PDT) Received: by mail-vk1-f175.google.com with SMTP id 71dfb90a1353d-487db6c9059so280852e0c.3 for ; Sun, 13 Aug 2023 13:31:14 -0700 (PDT) X-Received: by 2002:a67:e2ce:0:b0:443:7eba:e22c with SMTP id i14-20020a67e2ce000000b004437ebae22cmr5412726vsm.8.1691958674505; Sun, 13 Aug 2023 13:31:14 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 From: Gleb Popov Date: Sun, 13 Aug 2023 23:30:48 +0300 X-Gmail-Original-Message-ID: Message-ID: Subject: OpenSSL 3 ports fallout To: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-2.19 / 15.00]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; NEURAL_HAM_LONG(-0.98)[-0.976]; FORGED_SENDER(0.30)[arrowd@freebsd.org,6yearold@gmail.com]; NEURAL_HAM_SHORT(-0.22)[-0.219]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[arrowd@freebsd.org,6yearold@gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.173:from,209.85.221.175:received]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.173:from]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RP8Lr48jHz3cqV Some of the ports I'm using are failing to build after OpenSSL 3 import due to the following problem. OpenSSL headers that are shipped in base contain declarations of various deprecated functions for which libcrypto.so doesn't contain definitions. Some of them are RSA_generate_key and ERR_* family. These declarations aren't guarded by any #ifdef and are visible for ports software. VirtualBox and net-p2p/cardano-node detect these functions, try to use them and then fail to link due to undefined references. I believe this should be fixed in the base rather than patching each port? From nobody Sun Aug 13 20:46:59 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP8j244N2z4q9qn; Sun, 13 Aug 2023 20:47:02 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail2.karels.net (mail2.karels.net [3.19.118.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "freebsd", Issuer "freebsd" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP8j21gpnz3fpg; Sun, 13 Aug 2023 20:47:02 +0000 (UTC) (envelope-from mike@karels.net) Authentication-Results: mx1.freebsd.org; none Received: from mail2.karels.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail2.karels.net (8.17.1/8.17.1) with ESMTP id 37DKl0aE039574; Sun, 13 Aug 2023 15:47:00 -0500 (CDT) (envelope-from mike@karels.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=karels.net; s=mail2; t=1691959621; bh=+edzBFVtHZzCES1wPSGQ8c6auS6PVBfcOz7F3071tOE=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=XA3E9pLnndL5EpF8mliTnN6Z8Q1Hw14vyv4DPw+t5M/8lQ0c3rr2qUWXec2XuvtoK 24lWR+0nVy52TKm3grI9i2cSwpkGMKkJArjao+LbiMuSNT9yXwBSq71IqQQ2xJEjTQ OPa1Z5gQdUVJEHAxJf9sFftiVXbvD32AOdtsPUtwo8iuVBN493kJ2qeoOROKWC8I+B 2gwTvTY+36z7DQB+AAptuetzCCie8KwiR/y5d7RK0tlfabLU3Fz4XC8WTPNO7QB6Rh pKaToE3uuLkzMltt1xzRRjoao96ssHPXIcoQBtUrJdoa1G662b+NZpQD0Muz1oyTG6 UmbWSgOOSNzNQ== Received: from [10.0.2.130] ([73.62.165.147]) by mail2.karels.net with ESMTPSA id tpkRKERB2WSUmgAAs/W3XQ (envelope-from ); Sun, 13 Aug 2023 15:47:00 -0500 From: Mike Karels To: Juraj Lutter Cc: Mark Millard , FreeBSD Toolchain , Current FreeBSD Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 Date: Sun, 13 Aug 2023 15:46:59 -0500 X-Mailer: MailMate (1.14r5964) Message-ID: <277CBF0A-5620-433B-A2CA-045E5ABE3A82@karels.net> In-Reply-To: <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> References: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925.ref@yahoo.com> <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4RP8j21gpnz3fpg X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.16.0.0/14, country:US] On 13 Aug 2023, at 15:19, Juraj Lutter wrote: >> On 13 Aug 2023, at 21:13, Mark Millard wrote: >> >> But the offical CI builds on amd64 do not have the problem. >> >> We still have not found what is different about your context from the >> standard context used for CI builds and snapshot builds, all of which >> have worked find building on amd64 to target aarch64 with armv7 suppor= t. > > My src.conf: > > WITHOUT_PROFILE=3Dyes > WITHOUT_TESTS=3Dyes > WITHOUT_STATIC=3Dyes > > WITHOUT_LLVM_TARGET_ALL=3Dyes > > .if "${TARGET}" =3D=3D "arm64" && "${TARGET_ARCH}" =3D=3D "aarch64" > KERNCONF?=3DGENERIC-MMCCAM > REPODIR?=3D/data/poudriere/packages/pkgbase > MODULES_EXTRA=3D"rpi_ft5406" > WITH_LLVM_TARGET_AARCH64=3Dyes > WITH_LLVM_TARGET_ARM=3Dyes > .else > KERNCONF?=3DGENERIC-NODEBUG > REPODIR?=3D/data/poudriere/packages/pkgbase > WITH_LLVM_TARGET_AARCH64=3Dyes > WITH_LLVM_TARGET_X86=3Dyes > .endif I'm guessing that the problem is in the last section. When building to run on amd64, arm may not be included. I'd try adding WITH_LLVM_TARGET_ARM=3Dyes there as well. Mike > make.conf is empty, git workdir has `main=E2=80=99 checked out, without= any modifications (git status reports clean workdir) > > otis > > =E2=80=94 > Juraj Lutter > otis@FreeBSD.org From nobody Sun Aug 13 21:01:11 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP91k56lmz4qBkc for ; Sun, 13 Aug 2023 21:01:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP91k285sz4HJD for ; Sun, 13 Aug 2023 21:01:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691960488; bh=vCqZvvxcAQQ1hvlpzMF81TyOdpTA1qzoPfUOtcBgSD0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=RseubWXNUdzUJpBtDa/5KDND5jD6u4KkNKQO1itpPMZKxSCZgcGi4OLzXs2fRRyaxY968i7wtYZiwKTXKwhaB+O4j2VGmTc1BEPTElaVUX27cDwnGrJnCRJwR/uFEXhETBXvD7k3zSZxYvgxhe4MY9TmpU2FAY/fmEgb8VMRkpUDZSO4ptWQJchQ4HsJv/GUhv/ycSOSsfjpzfvMLVplAuGnLMKSc/hlqYpXMvyuuVO9IYTDid/tlNqFc/KrX3+8ipknI1faLjkSJ577PqT4QWdNpyTjzAqWn5CxwfjmwbcZ91TvXGXz+ZloQSNL5tTNwd/1S+3eKBvsuSFhKOv4JQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691960488; bh=UTIVjLhR4HkxEIrD08cSmQ1LZcbRhzK3Jg3lNG1e96u=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=UDcfaSe8JuaLtDhaEnutPw+cIPUsJDqVNr6N3n/6DFL9eQAj+AR7k4UsKqq3ZSg/srUCampooBmXsJApo482OpKguOxKDkp8Afk+CTmIOetrJ6d0rcMhtIj67XnlLjmu7zVvj58PWTegFmvWMSCti6H9LXKHBdJXDAO1CgmdUKjZsIAKE38ZQe39GEuKCtcF4gBc7NtBjFYq26bwNILhlvbhW20VmAyzzwJtbcrjLutemEEZJxeVONflHLvLM1wiVdQoDtr6wm5RU3nQByxmJ+WTJASN6PFZAjSj9kt/Ac0kOwy2uGdVCrHcFrMvfZGJdzSujH/j4LQ2DkFAjnIc9g== X-YMail-OSG: wQ5NvfcVM1kjiudZzZietMlLgae4Fy3nUEYsaG8wTVopvCt2d3oBd9_a_uH1RNT CQNmF9mluBc.jZjuvWQfhC_WGINtTBIqJPYe9F5xUm_x1bcRvLJGRpRnI5WjYsGzN6P1lOL6S535 G.Nr00lvmEopUk4WuOER.XoTeKjLZcWBmJGlRZGoTqrSu8BUZF7f8Jq18w7PMUSL0YsfgafwahKv MqDcWFZeQWIv.yTFznuJ_fl1gYyFH2aMoetVmn8W71eQ3vmMSUjDLpXRbSoYElw94KY3dwCdW3Lm tWPI.eVg5Fh16g9HUdKyQ5jX113WHWh9Z3qHzJHqzz5tuewv5v2l9D_7mMp3k54ItWQN0bcUZBWZ wmkjwd9UWqOCTYEy6O6OqQ2dL9_kKRze_H.UuU4Pfl9581oZVlFY43DgCFANFCNcRlo5jTHFx7ZZ 39cUnOnjmi9KO5zNS0B9Nf2vrTjMIs5ctLppy6nu0_9xw6Jw6DGBmLwjWyz2GKvD0o3rdaL6zmex 1Ry1K9PANEb1ltQZPyhED2L_LolRpN_.r_bTAgIyVr1za5RESt1d6Q_uwEXEpKyQpoqHyOz08flg 0ZBTcemUYBtaouC0rTbLDcseGHGJQffK_jWggEpoc1kkNk6X_JrIuPwgku1.YZFqKhQPgPnM2gcn oHzwJKBGcAq1yK8vvW.FunCX_G2Su.ErT6UXeUk0FDgs7gz5kfKzPdYqZJPJT89Fi4.ZKtQMBjby _COV.xBqP3Lf8o0AvjRP7GBN9VwFrDKf7wuGo18x2h2KLIG0iRxBvSVChtOfPeXP4mfsLX4VnEJj nyAZTp0jObPpsMPcAomVKT8Ie2y_AV_AIpJ.WsM.tbHKK0kvQcTDOfmfTmjI2GEsamrRoDac9Utk v7xZQEwqNSJKTDvUp1bWpeGHza2lGcDS90TkxDe6dNPLTR11mcQKzcZAsxRaPikifxamZ3rRdFVC wmK4jBDRjQ2EagVO.4j0BiuQaXMSmsNwWV8RCrX4rFJz2zFKO8Ozl84lrTBLUR2JCC.huMNay5yE hTPG7AGQkJE3_1fKbdG0CIKz4MNcD2PNR86U_xlbUuQ1KAyBXFy4TgCfn8WEWQdYP7dNwid32pe3 CYbTlzOD1M3TBItEWabUQHJ.YNtAMR6zeg7S1BOAQ5kPizkM1ywAvxLKFbqCZenl7PtMQyhAOw8N XfspCTS938WwIRsn3LzGxXS4Uj0VG8YUA8tKqu0w6VlzF74Sjx.ob8DjMVd6Z2E.tfVSIkHcOCnA 3k9Ki54AAbnbDjUyEPiQomY7ugwkAqitu1BuWKBnSEBmoghsa5acpDbfP0dRjPwYlgr0lccofW1I UYzdM2HXct0AU9MFJTEaZQVoJVbynjWUfveDIuZ2CCuuameV17A7F9eNTZRN_9ztT4a1SGar.ysM OeKGGTIa7ZR4cvAloAuLnhd_P5Cz3Yqf21nHf3L9m.RKIjVUp5ZraF_VGu6ZcFORHsi8W.FiPIZ1 1ZfHGdRJDRVtklGfd5sH0S3m7xYW9BAOnyq1838TrqnKEKhS9w8DFYZNeIZFqjlg4zVBbNaJK39G QilKqGcgZ.bung9CX8SMWuBbQtxF5ZOMj7GrUPlAMK9m9__R0IVwoE3rReNgsIoavzA8b_njSSu7 8wDgbAjMtMutIpztqUFDJSlRsZ4Yd90E8eMwbTtRTZ9oXgUhNQEyVzyiCU89dfBLziy2D6OaEK9w qO0pgglIGa2FQytjRwiyyeY2YZqMiNCYmjqwNQfVp5Eba3yqSnP3hzfpKh8g2lF1.n1_uKP1m8Us P.O9Y6wx1MtLc0_wo0D067XYeNf2DXaWSp0M7jNF1nrL03wbcq_1RwJWh3FW821lxR.hF4xUf4A0 BaKB5ppwng949Y4gNfbLH3tawWVYoqbrS6__RHXFM9Dk8U0VCeKnxDSXzeUwk5HUHO9tz1lRETD8 lV_iepItBGxnU0ZA0YAdet3uVyousde12dA0wY1evO6ZdbpCNXy.fsq6vroYngjPfZB9xQo8h1xE Fu6dvPjeNcjjuoFBvvFlc82u2.J0.xUhaXLz2KJU9xRlkyTRydYE6SQibr15gbUuMe955I8mRWjT OaZWRei1aERWjtU0kQS0XLIN_2ULLQX.utCWnvMewkhTIto.s7Zog0qLQVhp5gsFFo4VnsT.KLT3 XKuVygu0oSz3rJhOtJPPIq2ULJ2.JxbMvFUF4E4hnDy6JA.xmB5Hsix5CpTSaohA7SReEsBPI5KL HQaULFBxCuIN4J.OijpjD4lpO0tUbdUPmMuHYe6_F6pqfU9gMBWYisc83XpSrBdPKWro61BDvFsJ vtA-- X-Sonic-MF: X-Sonic-ID: 45da630d-e118-4db8-a9e1-4f276bf170df Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 21:01:28 +0000 Received: by hermes--production-bf1-865889d799-7vf9r (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 234c9fd11edbac4d54e82efdd7aaebf3; Sun, 13 Aug 2023 21:01:24 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Mark Millard In-Reply-To: <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> Date: Sun, 13 Aug 2023 14:01:11 -0700 Cc: Mike Karels , FreeBSD Toolchain , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925.ref@yahoo.com> <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> To: Juraj Lutter X-Mailer: Apple Mail (2.3731.700.6) X-Rspamd-Queue-Id: 4RP91k285sz4HJD X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Aug 13, 2023, at 13:19, Juraj Lutter wrote: >> On 13 Aug 2023, at 21:13, Mark Millard wrote: >>=20 >> But the offical CI builds on amd64 do not have the problem. >>=20 >> We still have not found what is different about your context from the >> standard context used for CI builds and snapshot builds, all of which >> have worked find building on amd64 to target aarch64 with armv7 = support. >=20 > My src.conf: >=20 > WITHOUT_PROFILE=3Dyes > WITHOUT_TESTS=3Dyes > WITHOUT_STATIC=3Dyes >=20 > WITHOUT_LLVM_TARGET_ALL=3Dyes Starting below this is wrong presuming all builds are actually done on amd64: > .if "${TARGET}" =3D=3D "arm64" && "${TARGET_ARCH}" =3D=3D "aarch64" > KERNCONF?=3DGENERIC-MMCCAM > REPODIR?=3D/data/poudriere/packages/pkgbase > MODULES_EXTRA=3D"rpi_ft5406" > WITH_LLVM_TARGET_AARCH64=3Dyes > WITH_LLVM_TARGET_ARM=3Dyes > .else > KERNCONF?=3DGENERIC-NODEBUG > REPODIR?=3D/data/poudriere/packages/pkgbase > WITH_LLVM_TARGET_AARCH64=3Dyes > WITH_LLVM_TARGET_X86=3Dyes > .endif Ending here. This is written as if there were separate toolchains for each target. That is not how llvm/clang/lld works, unlike gcc. One toolchain covers all the targets. That is what the FreBSD llvm support is structured for. You want the one llvm/clang/lld toolchain that includes everything required to build any/all of: A) amd64 B) i386 for amd64's lib32 C) aarch64 D) armv7 for aarch64's lib32 So the *_LLVM_TARGET_* parts would be more like: WITHOUT_PROFILE=3Dyes WITHOUT_TESTS=3Dyes WITHOUT_STATIC=3Dyes WITHOUT_LLVM_TARGET_ALL=3Dyes WITH_LLVM_TARGET_AARCH64=3Dyes WITH_LLVM_TARGET_X86=3Dyes WITH_LLVM_TARGET_AARCH64=3Dyes WITH_LLVM_TARGET_ARM=3Dyes So: outside the later conditional logic. (Your REPODIR?=3D lines are identical and could also be factored out of the conditional logic if they are not expected to be independently changed. Only the KERNCONF?=3D lines are actually target specific, needing to be in the conditional logic.) > make.conf is empty, git workdir has `main=E2=80=99 checked out, = without any modifications (git status reports clean workdir) >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 21:13:51 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP9JL3STVz4qCXl for ; Sun, 13 Aug 2023 21:14:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP9JK1Cxxz4Ln0 for ; Sun, 13 Aug 2023 21:14:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=p71GXC7C; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691961247; bh=uJw5nFDkRA9M1LrYsf2HW+JNlQdSdnEVaGS40+cI51A=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=p71GXC7Cup9/c7++w4KEDmZAy3+VLWW+tpoOD33oJVh+dp0E9jM1QbQSSd+t07k8qLWe66YCDhl7Kp5SqsU+cQeYWgspv1Mh4TKqx5uOXo/MFnYTsW9/ZiewghRh1TQHOZpl3NPwhQP4NEXktGtjZ2Oqw+QXchFhs1uSCkISF5CpR3uoRs5hjPdbCdl7q6x7DIbGNv0BMu70BGKa/4X5UH867Or1PEKFyxtLsrd90xyaJEA6Vk8f4RNWcjxWk7ixzFaZd5zvn/Hgi3G/HVC85+x3s9EadpAY1Xqa+jd7++JKCLKB0jzswi+ITrjQRfWIRK3ApR7FRZXoP9hQtlYbEg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691961247; bh=CItglXEWCl3zjPn7/bAsoCaqbJ7TWQ7hnH7oyjTNag+=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ZbBVPFEy9zWXCYpoawqvSliCvWrkY30RCwcfv6bQnYA0a0JAVGHeAAhh1CpP6cuNXOIJ/9RygmolL/0bpqYwJuHMLcpxo6NsKLY+P4bR38vcTItdcllEoXwIUlpazs7T70RpyjDAZaVOWTripf5gtp5kwXTwrT8EAKrHVfDvQgXAYGz63UDtWhhA76LbKEZ5eG+hJndcvkinlnOxor6sZ/W4ZQTR1I/MOKu7ViyVNtcCzTj1cLP3kLl9ptE+14RSqwD7+NA2HqsMF1/7re0BIacTfPmlEC0PaiZuIX5sUEC9TkkIMG2m2DGJ3K4pil0LDs4OA+5Jhg70hkuLhlMOtQ== X-YMail-OSG: q6ZN3tkVM1nopCKqpoSIxYg2g6BIWiRUJSicoklv18_t14XKJyF1hJedR2I.5vC MrqABOaLiALFalo0tgAyPpslXzPxswIlck6N9GpHcMWi6ugaUe4b4_Uw.qSOnz0TCQLS3_Y4MUq8 dM9PS6FlrAIT5sJL0qlZbTke5RrItc77PJOLDkoLKeZPKoFOiHjSSJ3jnB2k0VXn7rdwZPduOyIw pxtTI1SAyHZdmFViQ6e0nUDYoDe_niVXmsUM_0G.VfiVpFyzpx8Msc9wDBl3XRyJBdztiwaifXMd TGqC8iqioJn0riSwDpaA9cAggK4Whae5dmhHS6WYt5aRP889g3RwFkaDo5ouGHGVGBC638kduJoz 6KQAkQlbzd0FoPwgS.qiv5DdOPQOHiX0CW5f0W7poWmuJrOt.fP8MykARjzZrma7doVjAXh4CrFr 7WV2G6J0v31k.mNBbR2vgXspSTWQrZaPE8kn8einrthVmsF_ZwkBL7JuC_LijwJzE4vMFfLisZ.7 CeLVDSVB_FRHBxEIn1342BowHEdRLA_5B5DD.0J2GjzMNqlFy1hbDIlJGkK6qGSZ3_o.doMNpT7j 1WjA7Lr0MTKPObxpAZVEFvTMVz5HqJ5aVECutmXQnNSqTN5m74QseFKKJdhXKsnV2iwfSaHZ7Ox0 8rrk4dzbqLlblmzmGcLLpdwDCA.vzpUNvbqZJCSRCk4i1TFW.HEYM_uNTaHP53brddk2gh30UaEl gbgrzapg7mZyAjFf5MqROvxMUdoBOWRIS5SDzr74xyXVQVv08XDHlmPOtk38eMYS5RBOHjW9YBMR e4_D_Jh8m8_4QFJwSRlQ1iRi9M63R1bEZ7yGrrE9vFs2Bv2VlKqFhgqrj1SJIJvXWVdnaI.QfqSN Z0HmQC4mVXCh7qCj5AITzEG5nyjCZnSDdS0NETOG8GnlBiop0FNXBp_mv9sYEqaPAV8rsTNJNMxC VAfGk6ASpgrA_wmjXMt1oss5LxRXh1Hzzow53g.GMnaUKn_pzxuc8j1JYYOmmEpOA2FTCSmVV9me LGEadlmcPehHjZTNFxQrXKluHi5HpOiGmVBT5HVslaobRYCmWEaI92HTZi2xZPifr.le5_8lwUPe u_FVq8CysT4Q6OKIRyQLmxKfCmi0ZvTVfvR6OokG_7k.QfZ6L3Y54UsBto.XvZLZX_gZpT5Ts27q e9mlc8Re7bb98XJlHsfem.Av5wUCXnIjCm6gB_1q0kfjpBQcY2yM_uMo7p0BjQoKNQfZLMiHpuaF lXwssZjQdGOrU3REZoyUPsjuWK3DoJxOh83AByFIfeN7mzucnjG4ceJ2SeDttLPJoqHD1FXcIlTw liPLGf4LyuEwQ6AzP4JMLlgNuwDPikefLI9UeCCS4i2uQAHS6u.wkMKhyu_C0wNYN9AFO1OZnYA3 r8G_1NKpRhK01KzScr_ledPomJL2VFa.5U9_.dW8On2OmjFkgCj7tU8tjs9kkVyrY6YplWimItQu JIoDHXVyoOGsfBvEjRwatfnnlWQbVG6D2ZM3pf1P13Po8Em9N4pC1aV6kJCAeeBBta77yUTTR5gG 3jUv30zPA7Fr4mf4ylME3J3Pu6XgCMnqhDJJx81iktD812NZFvBkivRgyUQ78KRWJgGCiuxKhJHX E70VQbT8RwmYPBqWG_H.Kkxf53DjOlEt00ypZkdY44uMQVN0G6RBfkwgwO3rgfWkI7jeBXJeNM53 waldqKYZLMKQac7merYRL9tqZG.wZyWEAJdtAPuBuqmjckGsAXRpygRR1m0.WacoWxtCcK7upPBC uggBxcwI2sPzbfWvtpPzihJur2Mc67BAir57QyJiJsJvxz3_brcQgF7e5stQtcJhLdzMOK5sHrBx JuOcEfl_Vi2Nr.qckPWHsNI.c9rUuVmr6Ez_5ydg2ykoaI4lcVT0a6L30f.bfghdlxo69vB4L3GK WtxPBMqep7yUGxYMfKISbW5nzs1JjWoDx.3fT0E7E.0CZXzF6H5VcX1qqKyhf2WcrZbPuHpp1RlW 1ayG_2mtAQbUEHeZIOLfiwGGDlQr0aEUKzWA9NerqOrjZnSCg5Nnv.Z4wCcJv4r_U.mDDrIP7WVU 0I.HXoyFu6Nng5jsAOqk5wW_To_HX.z8BgJlppedxjMY8MQ00XksOxv6wsw_yvfM7NgRiGyyNfgv aWeSNh2ksxZWe5PyGIHywcGtYySp.PWqs8Kqfax.Q.VFTdoh0zEOSwLuxkiWmU0Xz3zJP5AhO2GI 2CYhEzSlnCH03pTjK81XpCjMtzCggbmXpiAG354Ji0ioSv6C2SSxl.LtCauD7wWAJns7HP2Xl4Fq 0eQ-- X-Sonic-MF: X-Sonic-ID: 71b70aec-fa2f-4bc2-ae6a-d6f6c7b4bfd6 Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 21:14:07 +0000 Received: by hermes--production-ne1-7b767b77cc-fbxjt (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2f9a19c08e2536d61c8059334967766a; Sun, 13 Aug 2023 21:14:03 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Mark Millard In-Reply-To: Date: Sun, 13 Aug 2023 14:13:51 -0700 Cc: Mike Karels , FreeBSD Toolchain , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <3C630E2F-959E-46BC-8DAE-DA19C3A6912E@yahoo.com> References: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925.ref@yahoo.com> <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> To: Juraj Lutter X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.148:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RP9JK1Cxxz4Ln0 On Aug 13, 2023, at 14:01, Mark Millard wrote: > On Aug 13, 2023, at 13:19, Juraj Lutter wrote: >=20 >>> On 13 Aug 2023, at 21:13, Mark Millard wrote: >>>=20 >>> But the offical CI builds on amd64 do not have the problem. >>>=20 >>> We still have not found what is different about your context from = the >>> standard context used for CI builds and snapshot builds, all of = which >>> have worked find building on amd64 to target aarch64 with armv7 = support. >>=20 >> My src.conf: >>=20 >> WITHOUT_PROFILE=3Dyes >> WITHOUT_TESTS=3Dyes >> WITHOUT_STATIC=3Dyes >>=20 >> WITHOUT_LLVM_TARGET_ALL=3Dyes >=20 > Starting below this is wrong presuming all > builds are actually done on amd64: >=20 >> .if "${TARGET}" =3D=3D "arm64" && "${TARGET_ARCH}" =3D=3D "aarch64" >> KERNCONF?=3DGENERIC-MMCCAM >> REPODIR?=3D/data/poudriere/packages/pkgbase >> MODULES_EXTRA=3D"rpi_ft5406" >> WITH_LLVM_TARGET_AARCH64=3Dyes >> WITH_LLVM_TARGET_ARM=3Dyes >> .else >> KERNCONF?=3DGENERIC-NODEBUG >> REPODIR?=3D/data/poudriere/packages/pkgbase >> WITH_LLVM_TARGET_AARCH64=3Dyes >> WITH_LLVM_TARGET_X86=3Dyes >> .endif >=20 > Ending here. >=20 > This is written as if there were separate toolchains for > each target. That is not how llvm/clang/lld works, > unlike gcc. One toolchain covers all the targets. That > is what the FreBSD llvm support is structured for. >=20 > You want the one llvm/clang/lld toolchain that includes > everything required to build any/all of: >=20 > A) amd64 > B) i386 for amd64's lib32 > C) aarch64 > D) armv7 for aarch64's lib32 >=20 > So the *_LLVM_TARGET_* parts would be more like: >=20 > WITHOUT_PROFILE=3Dyes > WITHOUT_TESTS=3Dyes > WITHOUT_STATIC=3Dyes >=20 > WITHOUT_LLVM_TARGET_ALL=3Dyes > WITH_LLVM_TARGET_AARCH64=3Dyes > WITH_LLVM_TARGET_X86=3Dyes > WITH_LLVM_TARGET_AARCH64=3Dyes > WITH_LLVM_TARGET_ARM=3Dyes Dumb editing error, not having an AMD64 line and having two AARCH64 lines. So, instead: WITHOUT_LLVM_TARGET_ALL=3Dyes WITH_LLVM_TARGET_AMD64=3Dyes WITH_LLVM_TARGET_X86=3Dyes WITH_LLVM_TARGET_AARCH64=3Dyes WITH_LLVM_TARGET_ARM=3Dyes > So: outside the later conditional logic. >=20 > (Your REPODIR?=3D lines are identical and could also be factored > out of the conditional logic if they are not expected to be > independently changed. Only the KERNCONF?=3D lines are actually > target specific, needing to be in the conditional logic.) >=20 >> make.conf is empty, git workdir has `main=E2=80=99 checked out, = without any modifications (git status reports clean workdir) >>=20 >=20 I should have noted that if the existing toolchain does not cover all of ARM64, X86, AARCH64, and ARM then it may take special activity to get to the point of having all 4 spanned in each instance of the toolchain. Once in place, things should be more self-propagating. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 21:37:56 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RP9r64v4nz4qFLX; Sun, 13 Aug 2023 21:38:14 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RP9r64McFz4Nrc; Sun, 13 Aug 2023 21:38:14 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691962694; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ui9DoWSYCUTsCrgR6iFfN4nqkzXPv7wy7sC69x/2nPE=; b=SjHrpcobC9P6mALGfpHXXnUTkAgzIzB6jRwgS1jTPh5SNRdIPP0OjkE9z18CWC32U5H0+E XA5ZW45fBzY+11Lw5vbIYUEwUA5zAyd+v7YxPuYYHO4k6RQIm9Wg9gJxLa+R41+jjTNXQQ S8AkCTw+8zAoP/fBEQDgVcJ/8E+m5S7LIH2JxQcON8k1qKjlvEFhZzhi8OirEk+ZWRiplp He5hU+hbsuKqVXEJRGfMc2QDac6n5UcxYxgvt+WKGU2vbHTlHGCn6OeoBWbhkVW2q3UtFK rwzObnAqbSsyF4vM/CSUZDKJgslbqHLlhQLzYDDfU2WF2swrCHX5p5OtMZL6sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691962694; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ui9DoWSYCUTsCrgR6iFfN4nqkzXPv7wy7sC69x/2nPE=; b=PYBDUsLFjGQIZSp2r+Kh3qXfHm96USSynUPTnXSwBQQhyOnKxDvTnXvfnv5VLBtV/CgBzQ Cum8flw1fH9C5QpyvRdkVtfYL0DkJyls1lCBdiYKfKMJ6Aa/W23OSmx9Ej7pneBylGidXr MQoiJ/z/nKl58I51WGy89qP+INE9LEhldC1MgfyGXcfVD1NHiRY3JlNKgDeKfXZKwUGNyy p4XYjj4PijtyHmFN8+4nPQ6f7kFYkBFjEIvGkUvoSeA1IKlv7BSwXXlAH6XfXcwQ34vaOu Xa+nW6+ri8Pg76FQ4cICG1ASeQT44jkHmFgvHAnPzbwGfd4V1TQnVfkjVSnFlQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691962694; a=rsa-sha256; cv=none; b=qi2G6VpwJloEoJJEGFAWIS9B7vx2ryRPQHx8Q4aFpP7Dc9WFH5YF6YJtQZO95HSAgdtkMI 4TiOjrUIfTaEl7M8A+G34GtZ1JW5BbvmYinaFp53697rj7iqQ2QPzt/CYRfi6sRK+sDqqP ax98h3xfD9/d1vzNnHubV3ulaARViNdiHuDh4FMb0AT8YeZs4OMQGb3YGn0L+JvN06nCdY ZqhQhoyn7zF3Dfhr+jU3HWYhy8BwgJzs7Baq9wh5qXMwYTWK770T6i5uNTox325K0DooK7 sxDHS5uCeCi6Ad4LgHsvNrrbA5lrZMHXj+WeYV7z0p5WzWQXNdPjnArlcPKmqg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ns2.wilbury.net (ns2.wilbury.net [IPv6:2a01:b200:0:1:f816:3eff:fecd:13e6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RP9r62dHHz150H; Sun, 13 Aug 2023 21:38:14 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 5A6AB61F91; Sun, 13 Aug 2023 23:38:07 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Juraj Lutter In-Reply-To: <3C630E2F-959E-46BC-8DAE-DA19C3A6912E@yahoo.com> Date: Sun, 13 Aug 2023 23:37:56 +0200 Cc: Mike Karels , FreeBSD Toolchain , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925.ref@yahoo.com> <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> <3C630E2F-959E-46BC-8DAE-DA19C3A6912E@yahoo.com> To: Mark Millard X-Mailer: Apple Mail (2.3731.700.6) > On 13 Aug 2023, at 23:13, Mark Millard wrote: >=20 > On Aug 13, 2023, at 14:01, Mark Millard wrote: >=20 >> On Aug 13, 2023, at 13:19, Juraj Lutter wrote: >>=20 >>>> On 13 Aug 2023, at 21:13, Mark Millard wrote: >>>>=20 >>>> But the offical CI builds on amd64 do not have the problem. >>>>=20 >>>> We still have not found what is different about your context from = the >>>> standard context used for CI builds and snapshot builds, all of = which >>>> have worked find building on amd64 to target aarch64 with armv7 = support. >>>=20 >>> My src.conf: >>>=20 >>> WITHOUT_PROFILE=3Dyes >>> WITHOUT_TESTS=3Dyes >>> WITHOUT_STATIC=3Dyes >>>=20 >>> WITHOUT_LLVM_TARGET_ALL=3Dyes >>=20 >> Starting below this is wrong presuming all >> builds are actually done on amd64: >>=20 >>> .if "${TARGET}" =3D=3D "arm64" && "${TARGET_ARCH}" =3D=3D "aarch64" >>> KERNCONF?=3DGENERIC-MMCCAM >>> REPODIR?=3D/data/poudriere/packages/pkgbase >>> MODULES_EXTRA=3D"rpi_ft5406" >>> WITH_LLVM_TARGET_AARCH64=3Dyes >>> WITH_LLVM_TARGET_ARM=3Dyes >>> .else >>> KERNCONF?=3DGENERIC-NODEBUG >>> REPODIR?=3D/data/poudriere/packages/pkgbase >>> WITH_LLVM_TARGET_AARCH64=3Dyes >>> WITH_LLVM_TARGET_X86=3Dyes >>> .endif >>=20 >> Ending here. >>=20 >> This is written as if there were separate toolchains for >> each target. That is not how llvm/clang/lld works, >> unlike gcc. One toolchain covers all the targets. That >> is what the FreBSD llvm support is structured for. >>=20 >> You want the one llvm/clang/lld toolchain that includes >> everything required to build any/all of: >>=20 >> A) amd64 >> B) i386 for amd64's lib32 >> C) aarch64 >> D) armv7 for aarch64's lib32 >>=20 >> So the *_LLVM_TARGET_* parts would be more like: >>=20 >> WITHOUT_PROFILE=3Dyes >> WITHOUT_TESTS=3Dyes >> WITHOUT_STATIC=3Dyes >>=20 >> WITHOUT_LLVM_TARGET_ALL=3Dyes >> WITH_LLVM_TARGET_AARCH64=3Dyes >> WITH_LLVM_TARGET_X86=3Dyes >> WITH_LLVM_TARGET_AARCH64=3Dyes >> WITH_LLVM_TARGET_ARM=3Dyes >=20 If you look closer, you will see that: WITH_LLVM_TARGET_AARCH64=3Dyes WITH_LLVM_TARGET_X86=3Dyes is in effect when I do =E2=80=9Cmake buildworld=E2=80=9D without any = TARGET and TARGET_ARCH, i.e. when I build the OS for the host (that is, = 14.0/amd64). But once I specify make TARGET=3Darm64 TARGET_ARCH=3Daarch64, only the = lines: WITH_LLVM_TARGET_AARCH64=3Dyes WITH_LLVM_TARGET_ARM=3Dyes are in effect. The line =E2=80=9CWITH_LLVM_TARGET_ARM=3Dyes=E2=80=9D has = only been added today for a test, but it did not make any difference. =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Sun Aug 13 22:11:48 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPBb96jH7z4qHH5 for ; Sun, 13 Aug 2023 22:12:05 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPBb95X46z4Rf2 for ; Sun, 13 Aug 2023 22:12:05 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691964724; bh=L5BM6LaDPNSIoulUb1X+rUJQwERAz8hucaQtdkK+2OI=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=ffLlsvZWLFxRe+5xNtVXAMcd1TgvdXqjLJV7RR+D0kNI+mTsstNqcHkPUZQE0EngqNe2i3g+o8Rl90w9itC/op2+VdkSGkW0LjQ/0a3FB9Ua9/CyeGJNxclzp9UaB5JTjH+ZwrqukHw4BwxJyhiR4nhBs4GxGI2lSVnJQ5mjKajKyXa6ZALniYG58278elJWXIQk9O/fS0/T7+MLq5upNN2diIHJcTAA/gybAPAC4Ff5NRFnoTe6enDxwwNzoAwJs2MSNvgYDeGjh0JUlx5scXDZucmtIleIdU7Dj+IbPBo4BAY0ruuzBX0Ty3mrGL6hCai485ELIlClQVAsqL9S9g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691964724; bh=3x3TFOky43UWTggZMyygg2fAwBS4HixQfrtE1Nu7W5m=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=luticV5oJgkgAhjseRjS3GQUG7qDzsjxpOoky/1NtecpzVaAsoyw700tmyKWOLAB8hToKEuTfI4iMVF39mOU+uoLx/Ja9gOjn5d3wCNo8nD915GP+CeH73Lr9TJ3v5KJ7bHaEmC/UH0EloCtN1j7F1eP32l9HfhpbUzd0SBeqApQ1bhMBrK+o4H3EZPfrSpNYQcMHcIQvRKZukdM5yAWvFfGaMSwhpJwN0bowwFTJ2KxzZfapqf1sIw0E9ZSr+HhEjV/isF8qNXyjc7m0lN99klt/GTq7OuSTOVQZg+ggblx/bkd1G/vqLYblbch799OxFhCa0whWhZtLp+eXz1HLg== X-YMail-OSG: 4fD5tuwVM1kuikiTM.qKqfBNPfngxAOKlQolsWgVb4mSS2TNY3FPFwf86GcXCO8 3GQTYhJ_7VNjGd39oOc3knyfzETtJy5sdTF47b5S5BAn81l7hnt_lTSD8jvHBm8QHjNW6hKryryq CBZup0VLA38b2vzyEJot0bNEeh2IiD6EnEX4Yp3pcpy5hs86A7NxL9GAddWGU5W5nYtCV9coh11f UO7w0675P6J3ZwhgZfbBoAp4I4Iu3L.kqpmRPx2I0QHaVDOmGwHfSBKX46eAX.Q2RLDLDGRc6X6K dOVkY7vNxWOLtTiQdeLyZ8uxiHLCfF3pUZ5HdLaLvltUEBPbP7JJDeIswUT5fchpRZriz_1MSCBp zgCWPD3b5Z55avXBLb7TPFpp194ryMBccK7X4YkQtiPpENWz2sDaXsdAmIKYWHLJTNr0ZOD2L4CT iuSxwbMOSukxvqm2D6tl30rh6ZlPMGyNC.X2KxdHdGLVPqCjv3oZ79Sy6wuVZG6OmHumQlklODj2 gYQB_nYrVuxnws8RStyg2JIx_FlJ0fl.upppOccBGK8OD1rhnHCx8EZXFRwfCdLxjmqDmQJdMrcI dymJ3eG7iLjn.pfpMChT1SL_zK5BfV2JPRqeoaWvv4ae.4eLpb13XQQ2HT22jfA_mJdwtgL.6WrK lqGtr3cHDof5K9eYIy3fJExhAtOxRG1qiccE96L6rHZABY3516jv_PsOqzWbck4ZWIqcCeU5BBgO Ig4d5kEq9hvL1mvkkWtMhnPpqAJh2JgxoROq8VT6qqYWu7OK0lbPOHAMqBF4f8KOuJPk4ZweiH51 9Iux9GWWj4MIX6TDOLmVu4T_iJt0iMgrxiEC2x52b7T_hmO70FZFj4gLROhwTA5EFJg2OHch9Fqq Gp_hQdJ1RJ4ItM9yogi5Izcnv7CZbIGcvQyN0DIzkTx.tbDx9fCiwjjOTRjyjtMKcYpbAT8TJWcX evL6So8wWkguQBQbHTsbgqF5oHA0E9avC1jylXorNoLXYgcoPqaHU8VZMaX1Zbh7SbtG0BmyR5VE oyyQVTEYVp1Ga92AUdd8hn0CIEJAOfp5n9KWTTk8hvkCalksTpI_0A0jgelkP.jecEU5IfHX0LCb QGj3Kmt3lKDliH_nx6SbMhTsgc_ksHJ35zxouViC9QZFRtPAz4w3_hQ7eI.fZqVo2dURnuJZNeCY j1UAT9BFYfzVtYBUCy5VEGfP80AZb5cKrxkcxkMtmGGCp1Cl75.5XPEocgFHV.gr.0bKYFFJ6juQ Y2i8w03COCl83P7.kG1mAFJS_0HQDlF7mdCpxJyqHYKvoa_xlNi.u3sghThEV2ULk5G08AXGrw6T 61OIf37cTaFD2gINys3ZDeoTqmHBXBj20eDKQr1HRrRMV_LqjnlSQevt6Wq.xy9uII.n3xhTyyeh idupYdmuWFIPmDz4.6DDu3nET2WUauc.jXNNyjkxNMMFHNul42cvWOSsw7uchkKNne8M2tptN0Bs 603CGShpC8SfJ8djLf0P9ohP3yo5gn0NJwhSrF1HXieO.8eTNew6fEzXVDckG1dr9dtsRTJd6TMT UhCBhbB7XBFYcSCaCFNnl0CTfG7t5oX5nI2m12DVfY62LmpP67bf13N1jw1E5h0EIYPyH.Faw6Bt onCeGW1KT2w7owMa8VMk3mJu7Z5fKL7Ye499lBZ.J6YHA.WKa3vMrh8psPk7wALuAAY2pKKvCvGC pd1auSb_eO.jBEkdN4vcz5rvtj9RH4j2un7U4cCnb5_JWGyUB08HVMJMOl68q1smwnJOJqGY91ra 3Hs_VtYuOPFi0DMZRQXARWQLlQBJngvSOUXol8p1GpeBVpVfySFc0CcbgSoZG52rqFkH1JpUddl1 2wUt3kcDJ3f3kM13BM8u6qapqfeP.8ZtWWtgOp2QITqE5Pnrg91n5ka4TcHdV42BK6buZes50e8y 2W7AIyBOg0n81I0BgJiWIryzt5R1JYqTYDjuLCkegrEIxTaHd3DSrKneY._2Dprt01VqSxDE8Fpf eLg1dWCXjqAGqmYf4PLCIMaxpla9esQe5NmFZ9zRTVhjFfNBup7mHo9QkNd6SOv2nA9B0cSDIUGF Zsxrx2SqUz7BGAb4LNxugsELfA73aJyA0PNyGo8DROXXp.RUayNXgXuyAWyqYHh7ZSKCebrVa06Q lx7jqtsgnP_w_fm9FI.zwtNAQ7mvVt.JvfwLqef3S6XU0KkBNyhAIkGxA_pXl3VetEB5dZUTaVwG x1dMH1CD.J8WdS9xyxrF7Ovrg_IqzkhdvJ3EV2HT10uwTp02yvwEyIyxw0BNF9Wg5_QuBOCCqo1_ b_g-- X-Sonic-MF: X-Sonic-ID: a3b3e44f-e94f-4330-af32-c84ba81fe1e4 Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 22:12:04 +0000 Received: by hermes--production-bf1-865889d799-7x4p2 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID e8e496810bd251cd28f0700eba45e041; Sun, 13 Aug 2023 22:12:01 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Error crosscompiling 14.0-ALPHA1 on amd64 for arm64.aarch64 From: Mark Millard In-Reply-To: Date: Sun, 13 Aug 2023 15:11:48 -0700 Cc: Mike Karels , FreeBSD Toolchain , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <3B0BBEB1-D16C-405A-B2FA-F53022CFC925.ref@yahoo.com> <3B0BBEB1-D16C-405A-B2FA-F53022CFC925@yahoo.com> <3AE647E3-B988-4387-BF56-A2DB6533B5FD@FreeBSD.org> <3C630E2F-959E-46BC-8DAE-DA19C3A6912E@yahoo.com> To: Juraj Lutter X-Mailer: Apple Mail (2.3731.700.6) X-Rspamd-Queue-Id: 4RPBb95X46z4Rf2 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Aug 13, 2023, at 14:37, Juraj Lutter wrote: > On 13 Aug 2023, at 23:13, Mark Millard wrote: >>=20 >> On Aug 13, 2023, at 14:01, Mark Millard wrote: >>=20 >>> On Aug 13, 2023, at 13:19, Juraj Lutter wrote: >>>=20 >>>>> On 13 Aug 2023, at 21:13, Mark Millard wrote: >>>>>=20 >>>>> But the offical CI builds on amd64 do not have the problem. >>>>>=20 >>>>> We still have not found what is different about your context from = the >>>>> standard context used for CI builds and snapshot builds, all of = which >>>>> have worked find building on amd64 to target aarch64 with armv7 = support. >>>>=20 >>>> My src.conf: >>>>=20 >>>> WITHOUT_PROFILE=3Dyes >>>> WITHOUT_TESTS=3Dyes >>>> WITHOUT_STATIC=3Dyes >>>>=20 >>>> WITHOUT_LLVM_TARGET_ALL=3Dyes >>>=20 >>> Starting below this is wrong presuming all >>> builds are actually done on amd64: >>>=20 >>>> .if "${TARGET}" =3D=3D "arm64" && "${TARGET_ARCH}" =3D=3D "aarch64" >>>> KERNCONF?=3DGENERIC-MMCCAM >>>> REPODIR?=3D/data/poudriere/packages/pkgbase >>>> MODULES_EXTRA=3D"rpi_ft5406" >>>> WITH_LLVM_TARGET_AARCH64=3Dyes >>>> WITH_LLVM_TARGET_ARM=3Dyes >>>> .else >>>> KERNCONF?=3DGENERIC-NODEBUG >>>> REPODIR?=3D/data/poudriere/packages/pkgbase >>>> WITH_LLVM_TARGET_AARCH64=3Dyes >>>> WITH_LLVM_TARGET_X86=3Dyes >>>> .endif >>>=20 >>> Ending here. >>>=20 >>> This is written as if there were separate toolchains for >>> each target. That is not how llvm/clang/lld works, >>> unlike gcc. One toolchain covers all the targets. That >>> is what the FreBSD llvm support is structured for. >>>=20 >>> You want the one llvm/clang/lld toolchain that includes >>> everything required to build any/all of: >>>=20 >>> A) amd64 >>> B) i386 for amd64's lib32 >>> C) aarch64 >>> D) armv7 for aarch64's lib32 >>>=20 >>> So the *_LLVM_TARGET_* parts would be more like: >>>=20 >>> WITHOUT_PROFILE=3Dyes >>> WITHOUT_TESTS=3Dyes >>> WITHOUT_STATIC=3Dyes >>>=20 >>> WITHOUT_LLVM_TARGET_ALL=3Dyes >>> WITH_LLVM_TARGET_AARCH64=3Dyes >>> WITH_LLVM_TARGET_X86=3Dyes >>> WITH_LLVM_TARGET_AARCH64=3Dyes >>> WITH_LLVM_TARGET_ARM=3Dyes >>=20 >=20 > If you look closer, you will see that: >=20 > WITH_LLVM_TARGET_AARCH64=3Dyes Looks like the above was my typing mistake: WITH_LLVM_TARGET_AMD64 ? > WITH_LLVM_TARGET_X86=3Dyes >=20 > is in effect when I do =E2=80=9Cmake buildworld=E2=80=9D without any = TARGET and TARGET_ARCH, i.e. when I build the OS for the host (that is, = 14.0/amd64). WITH_LLVM_TARGET_??? and WITHOUT_LLVM_TARGET_??? control building the LLVM/clang/lld toolchain. What matters is when the llvm/clang/lld materials are built and = installed, and when they are used. The way the builds work for building aarch64 and armv7 buildworld and/or buildkernel materials on amd64, the one-and-the-same toolchain is used thus inside one buildworld/buildkernel run: A) some amd64 files are generated that are later poteitnally run by the amd64 system. (So there is amd64 code generation involved.) B) lots of aarch64/armv7 files are generated that are not run by the amd64 system. (So there is aarch64/armv7 code generation involved too.) The same toolchain is used for both (A) and (B) inside one buildworld run. The toolchain has to support all those targets inside the one buildworld/buildkernel run. The whole type of analysis of avoiding having targets you intend to use not being in some of your toolchain builds is just inappropriate for FreeBSD and how it uses LLVM. Prat of the point of LLVM is to avoid having distinct tool chains (or toolchain builds) for different targets of interest. One toolchain is sufficient. You need to quit using the conditional logic for=20 ???_LLVM_TARGET_??? definitions if your builds are going to systematically work. What you are doing is not like what the CI or snapshot build so that work just fine. no FreeBSD documentatino suggests what you are doing. Make things work more like those and things should work just fine. > But once I specify make TARGET=3Darm64 TARGET_ARCH=3Daarch64, only the = lines: >=20 > WITH_LLVM_TARGET_AARCH64=3Dyes > WITH_LLVM_TARGET_ARM=3Dyes And that is a problem with what you are doing, givne your use of WITHOUT_LLVM_TARGET_ALL=3Dyes . > are in effect. The line =E2=80=9CWITH_LLVM_TARGET_ARM=3Dyes=E2=80=9D = has only been added today for a test, but it did not make any = difference. As I would expect. Make sure that all the targets you are going to build for have "WITH_" status all the time in the amd64 build system, not just sometimes. "WITHOUT_" status should only be for targets that you will never use when building via the amd64 build system. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 22:33:11 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPC3b24tmz4qJkl; Sun, 13 Aug 2023 22:33:15 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.zefox.com", Issuer "www.zefox.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPC3Z5KfJz4TjN; Sun, 13 Aug 2023 22:33:14 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Authentication-Results: mx1.freebsd.org; none Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.17.1/8.15.2) with ESMTPS id 37DMXBMJ025936 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 13 Aug 2023 15:33:12 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.17.1/8.15.2/Submit) id 37DMXBQl025935; Sun, 13 Aug 2023 15:33:11 -0700 (PDT) (envelope-from fbsd) Date: Sun, 13 Aug 2023 15:33:11 -0700 From: bob prohaska To: Mark Millard Cc: freebsd-arm , Current FreeBSD Subject: Re: alpha-1 armv7 git failed: fatal: pack is corrupted (SHA1 mismatch) Message-ID: References: <8DCC324C-966C-45DF-88F6-A442E348D4AA@yahoo.com> <2F080FD7-B63E-4FB6-A165-E24A3DF2730B@yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4RPC3Z5KfJz4TjN X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7065, ipnet:50.1.16.0/20, country:US] On Sun, Aug 13, 2023 at 12:45:12PM -0700, Mark Millard wrote: > > Wow. I'm going to suggest doing a clone (to a temporary > place) on one or more different types of system, such > as aarch64 or amd64. If, say, aarch64 works but armv7 > does not, then the corruption may well be in some armv7 > FreeBSD handling of data transfers, in places common > to both https:// and ssh:// use. > That seems to have worked on a Pi4 8GB running -current: root@nemesis:/usr # mv src src.old root@nemesis:/usr # git clone -o freebsd ssh://anongit@git.FreeBSD.org/src.git /usr/src Cloning into '/usr/src'... The authenticity of host 'git.freebsd.org (96.47.72.109)' can't be established. ED25519 key fingerprint is SHA256:y1ljKrKMD3lDObRUG3xJ9gXwEIuqnh306tSyFd1tuZE. This key is not known by any other names. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added 'git.freebsd.org' (ED25519) to the list of known hosts. remote: Enumerating objects: 4323641, done. remote: Counting objects: 100% (381285/381285), done. remote: Compressing objects: 100% (28204/28204), done. remote: Total 4323641 (delta 375527), reused 353081 (delta 353081), pack-reused 3942356 Receiving objects: 100% (4323641/4323641), 1.54 GiB | 390.00 KiB/s, done. Resolving deltas: 100% (3432012/3432012), done. Checking objects: 100% (16777216/16777216), done. Updating files: 100% (95944/95944), done. root@nemesis:/usr # > Note that, if you get a good clone, you can locally > copy the tree over to the armv7 media. But that is > not the point of my suggestion above. Under the circumstances it seems like the path of least resistance. Can I do something simple like sftp, using get -r ? Any trick to updating the copy? Many thanks! bob prohaska From nobody Sun Aug 13 22:53:58 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPCWn42BKz4qL6W for ; Sun, 13 Aug 2023 22:54:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-55.consmr.mail.gq1.yahoo.com (sonic308-55.consmr.mail.gq1.yahoo.com [98.137.68.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPCWm5rt9z4Wy2 for ; Sun, 13 Aug 2023 22:54:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691967249; bh=Wat27MP7HSgR3x4GkF0oFOJHeOhMso7sAvkPHkpK4A0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=JOjRzCxewympbS8BROygBZmGvuPm0QgsSBTbmI79RD/S6SmxAMmsB7/L/veK+l2TpBfUkxiDsLkdCEeyQDldefFPaMDCA9hEnNy7MTa2SwUooWYGjASR/HuVOj7VM2tac59ItZ5L77eyYTWnNhrHGpUccEfSVnXUAUWw2rRod/EAyCSJCx7QTOkqkD1AymcypU/AnpJhwFGWzNyg7iIy7CMnKMyMFBNViWikX/DtwRQOVylDAKUEmIx11uwI+nCjiwi/vhVbgNgmv/WsPE7AeCx/L9gku0fhc/kba0Hr9hume7rRo5ZAlTZMqhgfWYoAq0FrdPyd83yyzlacrfHpRw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691967249; bh=j0pG7Hp5SfYpSnkcvUmXVF5l32PCX0KYGh69yuanQKa=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=p4ECGuH1Qv8kuS1Ie6zV/kPjDjAMqPiF6JmBV7RRpNm70bsUJh0y6oJpaiuxUhKS8sBvTC+sSONkfiqOY81lrBQpebxWGE2Ok1DkXiPzuzyx3JfZYd2aDkWx+wvO4RiD7fin8c3mMLwByE0uwN538BGa/NCJ+4qD2ozC4n1RNIaMv8AeacBHODJ6MzlIy+ivbCDD5Ffb5As29x3wBEl1OlVvCOBz9mSQtWkPQlc1Ek4CsCFnjfeMJBxIJZa/09iqeDGOiefYYko7qnpYVg7JhaUBY0z/J33/aLtvceLhntmXtMYDbskP1KGQEHYJLwJ9wwZb4BZJhf0Y+In28qem5Q== X-YMail-OSG: _juLRxEVM1lQpQQryHmBnp3Un3JoRpPrvYe3sWMM.vI7E4l.Hzh1Il0jgsXvhXs vubCZQYzIuji.CIFqN9BWil_HPUs2Wqa9_jhQxR5.H16l8zFsAMoCu0BEX_6aZd4TQFiGA0FgEnG QAGgCSKBTwHBSwDu3gHwmJS9U6PkIzOXNxO4IGSnw9uWxfS3.CWFHQdjH1vHGhfdKWE4MEjqXdWh 52IxUlJZl1W8fXKr_eamLdi06EpSnisDyRADNOEZQB2hTnLzWCOv6RfZ4IDzryGqOJqVzC1.m_BE sWLtfUYqyYI_CHIbIXVOwiMaYJcqPtBliSTl15xmuHQldbH6PYp51SH8YAVHDUodzvnf07xRfE4a TaO92atSQAe22X6jTwDjiDgVhzWJB7dKR3e5I2Nu7rzm5IMavuNPuG93ilhzyCAG2RRbBT3Qa1.. ONDHdyz4Kis689dp6jxHFQKiKX2cbgAA2UF_dF22yji73uGMGcDfa6oqdX6DXnMWywxvCApUIo.s tAq2bqDS.Qktn8ZJDFV6eelFm_Nu51OxDAoSgv5Clv_k7Rfs9wIse6GeOxLFeOr5IrFXQ4RxyMUd XmjgSuWbrTR3GV2IAimrOWtfj7qNZUvLJ.R3ecjPWw0yrKXjtqV9vQ275sqt5yTF9WXyzCTGzEWw m0aU3FfH9U1CGNpY5qCJNY95yDpWsqz1.M6mkhTNdABLevZHjkO9PxFrmeZ1MZ2nCLCg26QuCcEE HnSxFX4TnwkSyX6VpZGsqpjkbocqG905umycGT7uCz0YzOTBO097WnK_mwhFdvwg6qgLOz9fhzZv oQscnj3qz3HrPiOmIfAs0znhAj8Qo7jIVzsD1sQhK0rauvn6C_GdG_axsTZw.8VkZrs_4qfweIon p3f2udidi4bYvetxY9DpBSQ5ArzitwGgPV_NOfxUxTK72GuyF2wc0sZhIa2_QSVJEr4mibut96IE _2UxjJV7XziFtfImIEamgjrZaGHGRSK.b2MfSvOrLI4sDDapJe_31e.VOyVdgsgsttzGqxPP1DSN WH5iTrG2PFbp2hqutuV7DWD31ahKLQw5MZ3dTTQXuEzgM3PsIGmDf50b3NjH3OwDdAJYYgBu_hdR wID5XVOaQ5bTZwtGbRE920bKuFp.yrOzKSLpVEsE2EtnDCdD4S6NZEhbycp2096mSavY1HQn1kHV 3KuEb6g7ptSg5iv_kSCFa1xCmTbrMPvW7OkGHrezsGLXB9Bk3DpgpDMgdreub4any0UlN2eeO2nr cYyiYVjxGhp3CnNERd0tX3wVLribqUrmUjHSKLadZxc8Tpq4VLwHrVvk3VA0M5b7Amvc2EjDpGX5 QbwU2FxmDUS1vB5IcXtGRQDjrf1pWgkByvhx.coLeDJl9KVnwjGxliwNFxwdkFdaBP6r7hgsizCL O1NcNBOiiJYhfw5cOaM9JknpwQpE3tKkFcxLJfULvpk8Tg2peP13Cp0Fr25HYNTAd7S4NDTNkd1K Ww8H_Yf2j2vSeg9PnwOK3BZP.2suhtEHruFWDOd9LZra.t32fB_dQM4QlT_pqK7p7zVXtElnWo7Y YcETV5o6mUD4pz8xoPvEx.7ns9Cb_vDYjIEGdegB.9H10IgqyBKNSj0y.KEu8t6Aj4rpT6DEB1zh qumejc3BebN9jkomWrO1EO_YBIn75iHF3z49lcwc31FvgxMXhfE1Uh8VG45oD9IRKTS9TsqBEcu8 BHhEC9m06e98agpG6wljwIfKvLfYFeQYvb1ZIhzPCuqP1POxfaGk0cjOhMNVlok0oafoFtoFkFKE Z7vVjsAZt48pPROu49EkATrxwgmbqzWwBoi0aEX5dGjpkllIEsb0ya_3SR9Q7FVT_UowvAYTsewD Y7N8UpuJ5faOdmRUBj.SU6yUTwcx_yLcolVtF26eOqqsJ9NeMsm1Ezq1yn0nmcbYcUNwtGisA4Jp WPnwt127eFYn2Mfrtg5v5XamG6zJWxKRQkJTKy9se6jp8tl5mDMJKvq1LJESu3ipzZYi7xWzqQec _8XY5AYKBdSepzcxaB829yE2YOogoM5JbhExNpqSKI3l12QtdQLyeq_MqdHSJ2ULLy7Bry0JISZT rME5nJFESXxYU4Vy6pTnndTi6jLupAYnsoseW7KsjDjt4oiTSHzAWlHaPKuweV7woV.XhdVjc0uU AgaNDDIE8eP54xVqJrO6cCAzGXExLVaFfh.dshlJfU78ZmlgE1wnhXvFhAciJEPmEfARPz3Vlser MFjvBaPnyNmcrIL.6ca9rrQGl9FaDLgoLEogPt7ghdIpTDbahmtjhkcho07YF_VUOsX0l4IIGmxP qGw-- X-Sonic-MF: X-Sonic-ID: c0cd8a2f-39ae-4e89-8b03-db968c165917 Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sun, 13 Aug 2023 22:54:09 +0000 Received: by hermes--production-gq1-6b7c87dcf5-rj4xx (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 93a1e15b1c659154832528f3622727ed; Sun, 13 Aug 2023 22:54:09 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: alpha-1 armv7 git failed: fatal: pack is corrupted (SHA1 mismatch) From: Mark Millard In-Reply-To: Date: Sun, 13 Aug 2023 15:53:58 -0700 Cc: freebsd-arm , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <0DE04A45-3D4E-4125-BD8D-221F30E3ACC7@yahoo.com> References: <8DCC324C-966C-45DF-88F6-A442E348D4AA@yahoo.com> <2F080FD7-B63E-4FB6-A165-E24A3DF2730B@yahoo.com> To: bob prohaska X-Mailer: Apple Mail (2.3731.700.6) X-Rspamd-Queue-Id: 4RPCWm5rt9z4Wy2 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] On Aug 13, 2023, at 15:33, bob prohaska wrote: > On Sun, Aug 13, 2023 at 12:45:12PM -0700, Mark Millard wrote: >>=20 >> Wow. I'm going to suggest doing a clone (to a temporary >> place) on one or more different types of system, such >> as aarch64 or amd64. If, say, aarch64 works but armv7 >> does not, then the corruption may well be in some armv7 >> FreeBSD handling of data transfers, in places common >> to both https:// and ssh:// use. >>=20 > That seems to have worked on a Pi4 8GB running -current: >=20 > root@nemesis:/usr # mv src src.old > root@nemesis:/usr # git clone -o freebsd = ssh://anongit@git.FreeBSD.org/src.git /usr/src > Cloning into '/usr/src'... > The authenticity of host 'git.freebsd.org (96.47.72.109)' can't be = established. > ED25519 key fingerprint is = SHA256:y1ljKrKMD3lDObRUG3xJ9gXwEIuqnh306tSyFd1tuZE. > This key is not known by any other names. > Are you sure you want to continue connecting (yes/no/[fingerprint])? = yes > Warning: Permanently added 'git.freebsd.org' (ED25519) to the list of = known hosts. > remote: Enumerating objects: 4323641, done. > remote: Counting objects: 100% (381285/381285), done. > remote: Compressing objects: 100% (28204/28204), done. > remote: Total 4323641 (delta 375527), reused 353081 (delta 353081), = pack-reused 3942356 > Receiving objects: 100% (4323641/4323641), 1.54 GiB | 390.00 KiB/s, = done. > Resolving deltas: 100% (3432012/3432012), done. > Checking objects: 100% (16777216/16777216), done. > Updating files: 100% (95944/95944), done. > root@nemesis:/usr #=20 >=20 >=20 >> Note that, if you get a good clone, you can locally >> copy the tree over to the armv7 media. But that is >> not the point of my suggestion above. >=20 > Under the circumstances it seems like the path of > least resistance. Can I do something simple like > sftp, using get -r ? Any trick to updating the copy?=20 >=20 Given that you are having networking problems, I was thinking more of moving the hard disk to be temporarily connected to the system with the good copy, mounting the UFS file system (I presume UFS), and using the likes of "cp -aRx SOMEPATH/ /mnt/usr/src/" to make the copy. (Presumes the armv7 media's /usr/src/ is empty.) Then umount, disconnect, and put things back on the armv7 system, boot that, and continue on. It is less obvious what to do about future git fetch activity on the armv7 system if that activity also has problems. But, you could try local area network based copying to explore if that also has problems. That would involve having git validate the copy after the copy is done. If you found that some methods work and others do not, that would be good to report. Similarly for the limiting conditions: all network copying fails vs. local network copies always work for all techniques tried. Up to you. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Aug 13 23:16:47 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPD1v0xncz4qMlN for ; Sun, 13 Aug 2023 23:16:51 +0000 (UTC) (envelope-from agh@riseup.net) Received: from mx1.riseup.net (mx1.riseup.net [198.252.153.129]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx1.riseup.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPD1s5sQpz4ZJB for ; Sun, 13 Aug 2023 23:16:49 +0000 (UTC) (envelope-from agh@riseup.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=riseup.net header.s=squak header.b=AIbE4WtL; spf=pass (mx1.freebsd.org: domain of agh@riseup.net designates 198.252.153.129 as permitted sender) smtp.mailfrom=agh@riseup.net; dmarc=pass (policy=none) header.from=riseup.net Received: from fews02-sea.riseup.net (fews02-sea-pn.riseup.net [10.0.1.112]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx1.riseup.net (Postfix) with ESMTPS id 4RPD1q5bvrzDqXQ for ; Sun, 13 Aug 2023 23:16:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1691968607; bh=KFpeZBnS/jGtg2TRorMqQaRfg0VbX9omf3NTGaxBTJc=; h=Date:From:To:Subject:In-Reply-To:References:From; b=AIbE4WtLzhOnZKdvLQU7SEFKnNVG5LNLNROQhWEuKxbbw3As99VsJNgGL1jl+xtZz rO7Q1FHB7L7AYzk2eanJuP4eINlnAVsjhvE1Y536/dSTCXAFMaqEIJ0nUH5+P8b0YQ pA9CHUfb9MxOQSyIfE39DIaqou0aOHtrY39e3lL0= X-Riseup-User-ID: FE1110278B1EAED402B12CBD4F4489DEB6CA0FEB63E66B98751C850524C72C69 Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews02-sea.riseup.net (Postfix) with ESMTPSA id 4RPD1q4V49zFsKp for ; Sun, 13 Aug 2023 23:16:47 +0000 (UTC) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Sun, 13 Aug 2023 23:16:47 +0000 From: Alastair Hogge To: freebsd-current@freebsd.org Subject: Re: Fwd: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 In-Reply-To: References: Message-ID: Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-4.20 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_LONG(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[riseup.net,none]; R_DKIM_ALLOW(-0.20)[riseup.net:s=squak]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[198.252.153.129:from]; RWL_MAILSPIKE_GOOD(-0.10)[198.252.153.129:from]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:16652, ipnet:198.252.153.0/24, country:US]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[riseup.net:+]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[riseup.net:dkim]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4RPD1s5sQpz4ZJB On 2023-08-13 23:22, Matthias Apitz wrote: > fwd to freebsd-current@ because I don't know if kde@ is read by someone; > > matthias > > ----- Forwarded message from bugzilla-noreply@freebsd.org ----- > > Date: Sun, 13 Aug 2023 15:08:04 +0000 > From: bugzilla-noreply@freebsd.org > To: kde@FreeBSD.org > Subject: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270041 Oh nice! How I have missed this BZ report, this will get my 14-CURRENT hosts up to date, thanks for sharing. From nobody Mon Aug 14 01:21:46 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPGpP2Shgz4TjvK for ; Mon, 14 Aug 2023 01:22:05 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPGpL5dJnz4jJ1 for ; Mon, 14 Aug 2023 01:22:02 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=OxbYtxIK; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691976119; bh=SUZ+qRKcrBC4gktXdAejWVlCFiaFcmL1TNRDJITZgoU=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=OxbYtxIKPfR6ptZ/iS9NyS5jDmSfsHAQ2HO4Byna3M6quHs58EDm62VVYcNLdEWsge+kaFyrlbYOkWLVrU4gkiQcI9co8+MP/ehtZQmZkOLdyaql3F+HrFgc/ApHhk8ADNOXL5ihX9AQDHU3TfC7EciTZG0RWxLWS6I+3xdegP2C/5vGj2BkelVDWzKcIg/HK582DoukZF5nH0M5kH/5UYVFVlRAF6breGodFIYSQdmEAmEE4CVrRl9M4jfQf2ki9KGfGKNj75KchfS56ao2nIeZBeIyfD++0S3t6LcJunF1s2P9QovcGUJQASMIe86zbdq/BhnhhmVD4KfpgQYFEg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691976119; bh=Aw3gDaZujC30K5IzVhsfCoo3LVWd2956YdbRMEFoKxQ=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=XdH0mME4r0q5W/AqMjWbd1cqsHzfLdbSQHsoH74Qp97HChFG7UdJ9ONHOs7EMaGIw85OB4eV6T0/2Wk9b86X3WSNarBgSx66XSP8MGW6k7Gjc3Lgv2x3pQRuD0SM9o863dQJIRXgxX60Vi+vYSLF28W0YbOKkp6RLCcizaB8p4cQGlg8/I47jj0Z1Te63afnTK59Zzx4IukN12FHd4cwWcPjpDfAv+dGpM0tgUyZYAHeSRCwRChiWcHUONAr8v7rv5jMKJ/ppkAXsWTvXLtmXjoYNpCYtHSiR8ZGbPI9kzG0RvDeBxUelnkAUcmkTGjeaLGvQsn5xAEXMBV2hLYwXA== X-YMail-OSG: eHXpPEUVM1liB7lqOiSfj3.jVvxzeFAc_tvNUBrjWGx9XjA2beqnpDkn6lMP7Be JhaikJmPGDKL3cX6Ic3tU8dcJGrWo0oWwS75adBNobyFUVJ_5sVG4H3ubNF0t0xCzC10N9aqW4SQ GRjI90oZEbFyF7TBS0LFhL.sKR6UNXr9et52TvTv43dBB.HyeYC0zYxfPUJkOlEEtgYx9WO0wsRy Pb9psHvzu5ksdNzKSHxVXEiQQVFpm_sbadz5_lrYVYh5clNiB91qc6nPgXJzbER9NcMhEzcABN8j BHI7zGpTV.XLtcDcrVLpjMxwM5JBLM3oHbFzu8ZSwyDogxOf7mONfF4Ed0DSj3DaRIow08OBE69X 7Z7FYVQXg9xXafBAVUDrUsS_4esmKRnfy0aOI2RHQExdpEeMz3CpCyUWt1mhHTEHWXe506ME1ExB Kcwmc63ZnMKtH8mMqLiYRNsFZPOevIK6u3iU.haDPIkk11AabAQ1kftp1cSiBfgbKdHTGeeZMJ.K m8tL.6t3f8PolxucnU2X3LV07JiCr3moZonUgQ6wmjN.baapOPPdeLgQawdqVWk88R521exxSkuk oWvAWpgOOluNw5AI6QkrOaPn3ife_GSlYNFwv7MFfjxRc71tPT_0lmod2jMB_WLdFZGx5lI6Aygp NpofLTdvdK4bnKZJvQFVRtXIWsdZsw3uvEdzeuz0BXVo9_H.30H1V69ni0qn4mZtc6YxFkDuULXH apeUEzyTtBYv.ULiJS.pMlQ_EwdbL_ZJMXLVJENIiyoiu20THnugQrqfcdusylbmd.EaetzbFG0K _1R5MIMyWBBkJRPq9ONr.LA0gVi9ifUdpql7E9bBgrDXEzXOTIfcxij5Pw1DnciQ5DDq6HK.YFaF s6cmnP36uzTStOlXPS6RQZT5mv08lMjdd71KR6PgnAPQslupwnurTc_E66lJQzm9UidHUW_oMP0B w6rh0VMRLXaRp6TtoXc4YY7MZfMr11diy83N4uxZvhHVOfc.KFAR4dTE.UDoW3p.0Kk.EQBMO5yn WV25YeNHiaxtM8a93ZqYbUnEFm6tzqMi0ktGJnvX0Y.vWeFll1eEeRtpuYEaq26Cr_sovIV5vQ2C ZZuA_y20AtuEYy0VwRVS3MvXyDmXd0wJf0O5uayt9bolXSJft_CQmahErvBurhzf1Iz.FpsXrgk. ouoFsWcqotIIDgXorVwV3afmGpsALJ0yXpmrTiOAHttLDv3n1_oGAHZyJnPvVLQ_VWGIhlW.LgJh wG5h0Fx.RWOWbuvvhT2d4sloCRMkLmaO_jdWuws7DKDhKCVXr3Aswwgt9GVs6.PgVJjTTRrUcVtk 0oXpk6mVL8SO1Q8sHGTRBGZOvPO2.ZzpwZQMJf8N8Km95GWH5_JMr9K96CJ7DWCiW.ehUPIm7GIG WI0KVnVh6j0xeQp5WnrMS2CY6tgG7uG9xkaHcs7Jy4XGAabjMdlkkVA_ZfUHQCf54lCXfQ9n_J6M laWSGFG1iGZaQUax8vKZTiNW4oXdcmCJZjve9BWDxFy9lLDmBLsGD8XG3ZQE3WqkIdgNAP80Vq7J d7NlQpquO0Q6IHFA3kmaLmPPxqWESA0DtxldYw5ouYspvLPmgPtmdT0IbqBenRgQrjeLnAf_LVob ECL33Z06cmJWzjaG32oRSpd5d2SQmWyMGtt_p7yS1QLFfWNXAfGKihMu05J83npcyvhVlzNuesyU 2jlxv9S8d_bIVX8_9oFuDQRJ5vIxcOrnmMgqwg7lA9sZgfo1mOErv9gbj56Fw_jgZyKCGfVL7hRT p3mVaybDnQglCjhZAPZTxES339IjuUDLePf1SCbApOC71oCGLpuqAJSIrdBFKJXto24G.ehhQkK3 E8GiW36wvUZO2JDVlOfvVZRgecuv_M4eReapTdgDXeDXXn6OconKrzRktXJqUjnKBoEN.3h89UOI EGa.KdtCgLqqUp4TX.yF8Wne1FzEA7FvYOlmOW34biq6A7g09oA0fUU9tXJP6h7d.fceKbBi2M15 sAuhKiy51jxUN7Xi4INdhNNXJXfNr8utNYEVAdk55y9VvokxsAyce6OLFdrvweDXcjmzQWYDeC.Z 1slccrgAJw2MgwDHHWw7f6SBBq1kq6VyecQvWz5VIXcauTEhWdC5RPY1K20DUdvFDsI6zWveWbAH 7nfn9_OPr.AK8gdnYI2NSaFkjVSU39A2Ik9S1W7aFkRgtlQyGHef4Tf6Kjrv_vlgXoP0XaKFNx4R N3bVUuUmKQKHiQH.MsXnHFZLgrL7Me9AqJ5YxDWxiJD4yBwQnvG9mrEFPFW9kEJtwmc2HZBSUqFC e55e6 X-Sonic-MF: X-Sonic-ID: efe00c56-2419-4654-b400-68e214c311f3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Mon, 14 Aug 2023 01:21:59 +0000 Received: by hermes--production-bf1-865889d799-2bcr6 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 371e047c8729859c717d64147d97104e; Mon, 14 Aug 2023 01:21:58 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: armv7 kyua runs via chroot on aarch64: zfs tests leave behind processes from timed out tests Date: Sun, 13 Aug 2023 18:21:46 -0700 References: To: Current FreeBSD , FreeBSD ARM List In-Reply-To: Message-Id: <6DBC3E10-29D9-4395-BD4F-65ACED87D821@yahoo.com> X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.45 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.95)[-0.948]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.83:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RPGpL5dJnz4jJ1 On Aug 3, 2023, at 00:19, Mark Millard wrote: > This is after the patch (leading whitespace might > not have been preserved in what you see): >=20 > # git -C /usr/main-src/ diff sys/dev/md/ > diff --git a/sys/dev/md/md.c b/sys/dev/md/md.c > index a719dccb1955..365296ec4276 100644 > --- a/sys/dev/md/md.c > +++ b/sys/dev/md/md.c > @@ -147,8 +147,15 @@ struct md_ioctl32 { > int md_fwsectors; > uint32_t md_label; > int md_pad[MDNPAD]; > +#ifdef __aarch64__ > + uint32_t md_pad0; > +#endif > } __attribute__((__packed__)); > +#ifdef __aarch64__ > +CTASSERT((sizeof(struct md_ioctl32)) =3D=3D 440); > +#else > CTASSERT((sizeof(struct md_ioctl32)) =3D=3D 436); > +#endif >=20 > #define MDIOCATTACH_32 _IOC_NEWTYPE(MDIOCATTACH, struct = md_ioctl32) > #define MDIOCDETACH_32 _IOC_NEWTYPE(MDIOCDETACH, struct = md_ioctl32) >=20 >=20 > The kyua run is still in process, but at this point there is > the following accumulation from the zfs testing timouts: >=20 > # ps -alxdww > UID PID PPID C PRI NI VSZ RSS MWCHAN STAT TT TIME = COMMAND > . . . > 0 17491 1 6 20 0 36460 12324 - T - 0:24.71 |-- = fsync_integrity /testdir2316/testfile2316 > 0 17551 1 5 20 0 10600 7512 tx->tx_s D - 0:00.00 |-- = /sbin/zpool destroy -f testpool.2316 > 0 17739 1 7 20 0 10600 7308 zfs tear D - 0:00.00 |-- = /sbin/zpool destroy -f testpool.2316 > 0 17841 1 3 20 0 10600 7316 tx->tx_s D - 0:00.00 |-- = /sbin/zpool destroy -f testpool.2316 > 0 17860 1 0 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 17888 1 3 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 17907 1 6 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 17928 1 7 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 17955 1 0 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 17976 1 4 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 17995 1 2 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18023 1 2 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18043 1 2 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18064 1 3 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18085 1 0 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18114 1 7 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18135 1 2 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18157 1 6 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18177 1 6 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18205 1 4 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18224 1 1 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18255 1 3 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18275 1 1 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18296 1 5 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18317 1 4 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18345 1 4 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18365 1 2 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18386 1 3 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18412 1 1 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18447 1 5 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18466 1 5 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18516 1 6 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18535 1 2 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade > 0 18632 1 0 20 0 10080 6956 spa_name D - 0:00.00 |-- = /sbin/zfs upgrade >=20 > Lots of these are from 300s timeouts but some are from 1200s or > 1800s or 3600s timeouts. >=20 > For reference: >=20 > = sys/cddl/zfs/tests/txg_integrity/txg_integrity_test:fsync_integrity_001_po= s -> broken: Test case body timed out [1800.053s] > = sys/cddl/zfs/tests/txg_integrity/txg_integrity_test:txg_integrity_001_pos = -> passed [63.702s] > sys/cddl/zfs/tests/userquota/userquota_test:groupspace_001_pos -> = skipped: Required program 'runwattr' not found in PATH [0.003s] > sys/cddl/zfs/tests/userquota/userquota_test:groupspace_002_pos -> = skipped: Required program 'runwattr' not found in PATH [0.002s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_001_pos -> = skipped: Required program 'runwattr' not found in PATH [0.002s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_002_pos -> = broken: Test case cleanup timed out [0.148s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_003_pos -> = broken: Test case cleanup timed out [0.151s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_004_pos -> = skipped: Required program 'runwattr' not found in PATH [0.002s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_005_neg -> = broken: Test case body timed out [300.021s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_006_pos -> = broken: Test case body timed out [300.080s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_007_pos -> = skipped: Required program 'runwattr' not found in PATH [0.002s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_008_pos -> = broken: Test case body timed out [300.034s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_009_pos -> = broken: Test case body timed out [300.143s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_010_pos -> = skipped: Required program 'runwattr' not found in PATH [0.002s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_011_pos -> = broken: Test case body timed out [300.003s] > sys/cddl/zfs/tests/userquota/userquota_test:userquota_012_neg -> = broken: Test case body timed out [300.019s] > sys/cddl/zfs/tests/userquota/userquota_test:userspace_001_pos -> = skipped: Required program 'runwattr' not found in PATH [0.002s] > sys/cddl/zfs/tests/userquota/userquota_test:userspace_002_pos -> = skipped: Required program 'runwattr' not found in PATH [0.002s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_001_pos -> = broken: Test case body timed out [300.052s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_002_pos -> = skipped: Required program 'labelit' not found in PATH [0.002s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_003_pos -> = broken: Test case body timed out [300.076s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_004_pos -> = broken: Test case body timed out [300.106s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_005_pos -> = skipped: Required program 'ff' not found in PATH [0.002s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_006_pos -> = broken: Test case body timed out [300.015s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_007_pos -> = broken: Test case body timed out [300.005s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_008_pos -> = skipped: Required program 'ncheck' not found in PATH [0.002s] > sys/cddl/zfs/tests/utils_test/utils_test_test:utils_test_009_pos -> = broken: Test case body timed out [300.051s] > sys/cddl/zfs/tests/write_dirs/write_dirs_test:write_dirs_001_pos -> = broken: Test case body timed out [1200.056s] > sys/cddl/zfs/tests/write_dirs/write_dirs_test:write_dirs_002_pos -> = broken: Test case body timed out [1200.046s] > sys/cddl/zfs/tests/zfsd/zfsd_test:zfsd_autoreplace_001_neg -> = broken: Test case body timed out [3600.055s] >=20 >=20 > . . . In more recent testing after the committed md_ioctl32 fix, including the 14.0-ALPHA1 snapshot: A) I no longer get the [300.015s] or longer timeouts. B) I instead get "failed: Setup failed" in under 1 sec for those test cases. C) I no longer get any hung up processes being left behind. (C) is likely expected, given the (A) -> (B) change. It does not look like this report will ever progress to a bugzilla submittal, given what is observed in the 14.0-ALPHA1 snapshot. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Aug 14 01:55:04 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPHXc2WWBz4TmK2 for ; Mon, 14 Aug 2023 01:55:12 +0000 (UTC) (envelope-from grog@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPHXc16jdz4mfL for ; Mon, 14 Aug 2023 01:55:12 +0000 (UTC) (envelope-from grog@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691978112; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=Awy1ad/jjMrpjp/z1mCXsR3w+7Ed180qH+6vljg0g7I=; b=PQPcRf7PS/l6x73+0lvv43z27Zd0s3Qof3ovKXKf3PeMdmaZ6zSSPRU6Jap8S2Wp0+ym7B Dt/6I/ZvyBpIR0S/ZdZfksvVdR5on2X/SmGga/lIuXnvTj2L+6ruG9s4ozgSTH1o+s45wI vm3Wsx5J7XG0SKeGBSB+sLIqLz4XXUHv6eHIghUTtBNAO9nz73KwPH6WKXzag72zDBGGWB Q4wIlDG/+moRrsq3vjhdK7903eiYDEjxaUSgna2DHGDkItvroXdvpa7QcChDJjjyo5JP7b GXmmWQZV5hDuPKz0cdsXlmJ4+iA7fJh2MWyLobejP1D6QsXGXVBd4N6Uiin2sg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1691978112; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=Awy1ad/jjMrpjp/z1mCXsR3w+7Ed180qH+6vljg0g7I=; b=vn2IbCZ+FzdXWFf4+cruNlLhYOox3oOwaFzNg1Ni/mNTgbLLCsFVN+ye6CdaVRcOVg8K1r jcF3O+gT18j1KmVKj9uqNWoV3zfimN9/rColC6TtpWw/eAnu+4T5xtBMvBHwaMfJJqwRZC tdvCaCwgT/D1YIYju9j5IFi4UQ6754HuPXPdBJ9St5FKK8KnnYVDYbLSpRtDFoR6e2EhJh oo8JKuNo+4EpVpfGRT4OF9qgFCR4ucge4my0lzgE3AUxbsdgWBLRR9Dc23Sxh8aCJfkdrh bVdUmdAlSUwgPtT91D5Cf8QwpMLf0lEaN2TjMxszKUpweHlc8hOI/Z+v3NVLmw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1691978112; a=rsa-sha256; cv=none; b=K8wX5EsLLP5pGZPl2sc3vjsFQyt9UFIO02lpXmZTpMxlM00Su3ZUjUKOqeii4/orf/HEq0 QbOGPyDV6Jrm3XsjWxwuUw46C6zBanqh3ut2vmyd+zIM0KDixuUBvtrDGZKQ2UsJVu9/0b ZgoC2tx2ZApTE7miEBf4AI3aS55zSXrtFhzpcjyy0mFKWbYyN+YeKUNiRFXGFwBUW1caeD q0zt8CdiLHZh+IgWtCgfC8MEiKa9FKKQ0q9k8ksUV/CCjLfYVSbzKPhSQC7kSyzfcdQ71w AhJTfwu0RCVMCLqvkbRv/d0uODUJtBuhxvZZXbfGZLdwmrc0SN5TV/V8rAPtNw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from eureka.lemis.com (121-200-11-253.79c80b.mel.nbn.aussiebb.net [121.200.11.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: grog/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RPHXb1Qlxz17cR for ; Mon, 14 Aug 2023 01:55:10 +0000 (UTC) (envelope-from grog@FreeBSD.org) Date: Mon, 14 Aug 2023 11:55:04 +1000 From: Greg 'groggy' Lehey To: current@FreeBSD.org Subject: Potential show-stopper in em driver? Message-ID: <20230814015504.GB87066@eureka.lemis.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3uo+9/B/ebqu+fSQ" Content-Disposition: inline Organization: The FreeBSD Project Phone: +61-3-5309-0418 Mobile: +61-490-494-038. Use only as instructed. WWW-Home-Page: https://www.FreeBSD X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 User-Agent: Mutt/1.6.1 (2016-04-27) --3uo+9/B/ebqu+fSQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I've spent the last couple of days chasing random hangs on my -CURRENT box. It seems to be related to the Ethernet driver (em). I've been trying without much success to chase it down, and I'd be grateful. The box is headless, and all communication is via the net, which doesn't make it any easier. I've tried a verbose boot, but nothing of interest shows up. Typically it happens during the nightly backups, which are over NFS: Aug 13 21:06:46 dereel kernel: <<<66>n>nffs server s6>neurekfs server aeureka:/dum:p: /ndoump: tnot responding Aug 13 21:06:46 dereel kernel: Aug 13 21:06:46 dereel kernel: responding Aug 13 21:06:46 dereel kernel: Aug 13 21:06:46 dereel kernel: server eureka:/dump:n not responding And if you haven't seen those garbled messages before, admire. They've been there for a long time, and they have nothing to do with the problem. More to the point, there are no other error messages. I've run three kernels on this box over the last few weeks: 1. FreeBSD dereel 14.0-CURRENT FreeBSD 14.0-CURRENT amd64 1400093 #10 main-n264292-7f9318a022ef: Mon Jul 24 17:13:32 AEST 2023 grog@dereel:/usr/obj/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC amd64 This works with no problems. 2. FreeBSD 14.0-CURRENT amd64 1400094 #11 main-n264653-517e0978db1f: Thu Aug 10 14:17:13 AEST 2023 grog@dereel:/usr/obj/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC 3. FreeBSD dereel 14.0-ALPHA1 FreeBSD 14.0-ALPHA1 amd64 1400094 #12 main-n264693-b231322dbe95: Sat Aug 12 14:31:44 AEST 2023 grog@dereel:/usr/obj/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC amd64 Both of these exhibit the problem. Note that we're now ALPHA1, so it's a good idea to get to the bottom of it. The box is an ThinkCentre M93p. I'm attaching a verbose boot log, though I don't expect anybody to find something of use there. I'm also currently building a new world in case something has happened since Saturday. Greg -- Sent from my desktop computer. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA.php --3uo+9/B/ebqu+fSQ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAmTZiXgACgkQIubykFB6QiMnWQCdFU/dW/vMkLh7g0ErYzi4tGXy mQwAniPoXpfiG0O4qs2q9H8IsflmVV7T =zXjN -----END PGP SIGNATURE----- --3uo+9/B/ebqu+fSQ-- From nobody Mon Aug 14 03:42:22 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPKwQ6sd0z4TvPm for ; Mon, 14 Aug 2023 03:42:30 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPKwN5Zjnz3D03 for ; Mon, 14 Aug 2023 03:42:27 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp; dmarc=none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37E3gMHR011674 for ; Mon, 14 Aug 2023 12:42:23 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Mon, 14 Aug 2023 12:42:22 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: Re: Fwd: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 Message-Id: <20230814124222.1cdee829aaaaf0bb17ca3e5f@dec.sakura.ne.jp> In-Reply-To: References: Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [0.42 / 15.00]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.92)[0.924]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[sakura.ne.jp]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Spamd-Bar: / X-Rspamd-Queue-Id: 4RPKwN5Zjnz3D03 On Sun, 13 Aug 2023 23:16:47 +0000 Alastair Hogge wrote: > On 2023-08-13 23:22, Matthias Apitz wrote: > > fwd to freebsd-current@ because I don't know if kde@ is read by someone; > > > > matthias > > > > ----- Forwarded message from bugzilla-noreply@freebsd.org ----- > > > > Date: Sun, 13 Aug 2023 15:08:04 +0000 > > From: bugzilla-noreply@freebsd.org > > To: kde@FreeBSD.org > > Subject: [Bug 270041] www/qt5-webengine: Poudriere build fails for 5.15.8 > > > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270041 > > Oh nice! How I have missed this BZ report, this will get my 14-CURRENT > hosts up to date, thanks for sharing. Maybe because it was once closed. :-) Now it's reopened. Feel free to pop in and report your result there. Applied the uploaded patch, or increased ulimit in poudriere.conf. Update OK or not. -- Tomoaki AOKI From nobody Mon Aug 14 03:44:14 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPKyj02qRz4Tvg1 for ; Mon, 14 Aug 2023 03:44:29 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPKyh581kz3FCy for ; Mon, 14 Aug 2023 03:44:28 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102e.google.com with SMTP id 98e67ed59e1d1-26b41112708so1040228a91.3 for ; Sun, 13 Aug 2023 20:44:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; t=1691984667; x=1692589467; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=FvQ4dprH7EMStJL9isHR+vY2gyhC6JY9tlVyLmvy/EU=; b=SNikjMSl9R4R3fRPIhX2c1zMIaBHdViwPkTognK0nP2Sy/XvuAQINrSZS7UNguzEA+ /JPlPOiXpIRN/A7Z3MCvu9iOHwhI3IK+NuMUSnV8CXwIiu+ndNUXEN7JCMP/jMf+LfVf ktEbFDu4Dre8N/RDQCbmYSLH9OpzpecA/4L/U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691984667; x=1692589467; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FvQ4dprH7EMStJL9isHR+vY2gyhC6JY9tlVyLmvy/EU=; b=AjNX1davbTOt3XrO99elAW71DGP95bcxOydPblfv/1JogX9JMp6iRkTfxpMtdiuTeF l9BcRouWkPzFYkWWS9n1Fn7ak1jLk9wU1X+jSxyrOfAUw+Px9ZEHDs/BIvqY7cxh9vDa USjNed34DjORiS4t5Mzpi9wbgCJjBjJG0a/mqawzQkfQD6Z0sD5gp+AItKGebT1kE9kP TQuy+iBFDSG75dR4nb6qGFUIcbI8RR5wH59dXbAoj1OY6QtS3jA934HEWdaF7NI7ttBy S8KDMZVhkr1mE0W6s7VSYg/GuOVBku0TMA8GDynEPkb8sFFHLnq1g/ipqrrw/Qf5s0Lg KSAg== X-Gm-Message-State: AOJu0YzD2kgIzevD2pMJ5oUDALbyKp+csEJ6h4L31VLcnhEVbMS6qMek vUWgGdnU8vQWKv9STpB7x+8g9O6f4+pqtpAPfyix9G6y1t/Ly8w8 X-Google-Smtp-Source: AGHT+IFLbB6PCG7l8O8xb6hLWx+Kb/FLi+pklcA7sV9z1c9hvnrmbobesX5TxPsf7fwCvaCpMsL9MBnFBgLw/1ngztk= X-Received: by 2002:a17:90a:9501:b0:269:32d2:5ac4 with SMTP id t1-20020a17090a950100b0026932d25ac4mr7572966pjo.25.1691984666754; Sun, 13 Aug 2023 20:44:26 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <20230814015504.GB87066@eureka.lemis.com> In-Reply-To: <20230814015504.GB87066@eureka.lemis.com> From: Kevin Bowling Date: Sun, 13 Aug 2023 20:44:14 -0700 Message-ID: Subject: Re: Potential show-stopper in em driver? To: "Greg 'groggy' Lehey" Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4RPKyh581kz3FCy X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On Sun, Aug 13, 2023 at 6:55=E2=80=AFPM Greg 'groggy' Lehey wrote: > > I've spent the last couple of days chasing random hangs on my -CURRENT > box. It seems to be related to the Ethernet driver (em). I've been > trying without much success to chase it down, and I'd be grateful. > The box is headless, and all communication is via the net, which > doesn't make it any easier. I've tried a verbose boot, but nothing of > interest shows up. Typically it happens during the nightly backups, > which are over NFS: > > Aug 13 21:06:46 dereel kernel: <<<66>n>nffs server s6>neurekfs server a= eureka:/dum:p: /ndoump: tnot responding > Aug 13 21:06:46 dereel kernel: > Aug 13 21:06:46 dereel kernel: responding > Aug 13 21:06:46 dereel kernel: > Aug 13 21:06:46 dereel kernel: server eureka:/dump:n not responding > > And if you haven't seen those garbled messages before, admire. > They've been there for a long time, and they have nothing to do with > the problem. More to the point, there are no other error messages. > > I've run three kernels on this box over the last few weeks: > > 1. FreeBSD dereel 14.0-CURRENT FreeBSD 14.0-CURRENT amd64 1400093 #10 mai= n-n264292-7f9318a022ef: Mon Jul 24 17:13:32 AEST 2023 grog@dereel:/usr/= obj/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC amd64 > > This works with no problems. > > 2. FreeBSD 14.0-CURRENT amd64 1400094 #11 main-n264653-517e0978db1f: Thu = Aug 10 14:17:13 AEST 2023 grog@dereel:/usr/obj/eureka/home/src/FreeBSD/git/= main/amd64.amd64/sys/GENERIC > > 3. FreeBSD dereel 14.0-ALPHA1 FreeBSD 14.0-ALPHA1 amd64 1400094 #12 main-= n264693-b231322dbe95: Sat Aug 12 14:31:44 AEST 2023 grog@dereel:/usr/ob= j/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC amd64 > > Both of these exhibit the problem. > > Note that we're now ALPHA1, so it's a good idea to get to the bottom > of it. The box is an ThinkCentre M93p. I'm attaching a verbose boot > log, though I don't expect anybody to find something of use there. > I'm also currently building a new world in case something has happened > since Saturday. The verbose boot didn't make it in the email. 'dmesg | grep em' would be a good enough start. Can you post 'sysctl dev.em.' from the machine after a lockup? Since you don't seem to have OOB access you may have to get creative with cron or something. > Greg > -- > Sent from my desktop computer. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft mail program > reports problems, please read http://lemis.com/broken-MUA.php From nobody Mon Aug 14 03:47:03 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPL1l4svpz4Tw6l for ; Mon, 14 Aug 2023 03:47:07 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPL1l1gVyz3GcL; Mon, 14 Aug 2023 03:47:07 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37E3l4a6012299; Mon, 14 Aug 2023 12:47:04 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Mon, 14 Aug 2023 12:47:03 +0900 From: Tomoaki AOKI To: "Greg 'groggy' Lehey" Cc: current@FreeBSD.org, Kevin Bowling Subject: Re: Potential show-stopper in em driver? Message-Id: <20230814124703.3875a7ba266cdf079525eb82@dec.sakura.ne.jp> In-Reply-To: <20230814015504.GB87066@eureka.lemis.com> References: <20230814015504.GB87066@eureka.lemis.com> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4RPL1l1gVyz3GcL X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] On Mon, 14 Aug 2023 11:55:04 +1000 "Greg 'groggy' Lehey" wrote: > I've spent the last couple of days chasing random hangs on my -CURRENT > box. It seems to be related to the Ethernet driver (em). I've been > trying without much success to chase it down, and I'd be grateful. > The box is headless, and all communication is via the net, which > doesn't make it any easier. I've tried a verbose boot, but nothing of > interest shows up. Typically it happens during the nightly backups, > which are over NFS: > > Aug 13 21:06:46 dereel kernel: <<<66>n>nffs server s6>neurekfs server aeureka:/dum:p: /ndoump: tnot responding > Aug 13 21:06:46 dereel kernel: > Aug 13 21:06:46 dereel kernel: responding > Aug 13 21:06:46 dereel kernel: > Aug 13 21:06:46 dereel kernel: server eureka:/dump:n not responding > > And if you haven't seen those garbled messages before, admire. > They've been there for a long time, and they have nothing to do with > the problem. More to the point, there are no other error messages. > > I've run three kernels on this box over the last few weeks: > > 1. FreeBSD dereel 14.0-CURRENT FreeBSD 14.0-CURRENT amd64 1400093 #10 main-n264292-7f9318a022ef: Mon Jul 24 17:13:32 AEST 2023 grog@dereel:/usr/obj/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC amd64 > > This works with no problems. > > 2. FreeBSD 14.0-CURRENT amd64 1400094 #11 main-n264653-517e0978db1f: Thu Aug 10 14:17:13 AEST 2023 grog@dereel:/usr/obj/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC > > 3. FreeBSD dereel 14.0-ALPHA1 FreeBSD 14.0-ALPHA1 amd64 1400094 #12 main-n264693-b231322dbe95: Sat Aug 12 14:31:44 AEST 2023 grog@dereel:/usr/obj/eureka/home/src/FreeBSD/git/main/amd64.amd64/sys/GENERIC amd64 > > Both of these exhibit the problem. > > Note that we're now ALPHA1, so it's a good idea to get to the bottom > of it. The box is an ThinkCentre M93p. I'm attaching a verbose boot > log, though I don't expect anybody to find something of use there. > I'm also currently building a new world in case something has happened > since Saturday. > > Greg > -- > Sent from my desktop computer. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft mail program > reports problems, please read http://lemis.com/broken-MUA.php Does commit f1b5488f7bba7f25a57750f87cbcbccbd5b9d16b [1] related? How does it go if disabling TSO as documeted on the commit message? CC'ing kbowling@, who committed all (I could find on cgit) e1000/em-related bits between the timeframe of 1. and 2. [1] https://cgit.freebsd.org/src/commit/?id=f1b5488f7bba7f25a57750f87cbcbccbd5b9d16b -- Tomoaki AOKI From nobody Mon Aug 14 04:04:29 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPLQ91Njsz4TxQ3 for ; Mon, 14 Aug 2023 04:04:49 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPLQ80ljgz3JTL for ; Mon, 14 Aug 2023 04:04:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=miSPlm0g; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691985885; bh=z/MhyCPGlS6u2PHxGxd4CUJ6rx4c3OgCz9N6dQoo7jA=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=miSPlm0g4ySOk3wwSxuFVquX0lnKOgb10+Mrr0u83iBexi6VotanMS4JLHQHGv4XE4pE4OjraR8LJ0IyEN6+VNA0yHzE6t/7jV6RVp5NPJTATdriPh8znpj4B6CIMjKnHcd7gzTfCIZZSZFyXKe7ny++tmbrkrJekquLaOzUXzDoplUyv4wyL4IkXnBsTejO+MfIygCDL+0qdC5RqIqi3v4j7X8s6xUv3A1vQ4FeAvyURbTi1FqCqoICU4lTJ/X3zXXviDqv9hdS+VARi/NGp4lJQoVD76oADQAZiQbDmAWldthqhM6Ci7joyC6IMyZ4EVXoCUOO5+8JhUegMo6gyA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1691985885; bh=G9KpGSw7lAEmwhinXIH/rkLukL2rEjCAv/Udcsm32O7=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=srGtNhdz/3cCqt2Fh0RDGTp+HC+iELomNidXKpznbLXKSZN4oa9qB2zhNazbmeTKiMmGE8/86XnuQOeApFKLd1NoBWS8F08rAjUeRLtvKDl95Qf9h1WvgRNm1Kyl3OJWshbLymAKNF/1rc2filC4maz5WDJrib8zIQlDt+RWlR4aBnMe5VH+50YH4ip7PVCZsClvaabhKKe3IMv+BmiQQpNu+PmBSwXGh9IqdUpuPclN2YOLQbR8bQ1xlwITGrNX2Zbi6cBb+LY8icwEeRvd+E4bcgHEqmhAktG9A5tf5W4xqtRmzEZO0WaVh3ZGcBFvNlYSrmq4Z7CzCrp28T0euw== X-YMail-OSG: yRzYNBwVM1mg8jCMi8n6neV4OjaA1x7rcgra5_wRV0DRffrFom_rutEKVYanK6T jI8RzyiRNWlbfr7TvMIjmt36QO30VcW6koxLCExj2UqIqxAXKBB0B8mttV8A1eYqNVlTwbkLx_oo 8RKyETuodhNKX_3FwBSThCivdPKgwiFgUjE26QaL3_sFv0ulAmrfJE9dp7NAh9pumqz9iii4l53c wZ02.GxZ_U8CN4TuzsBUQpAvsZrB5hW9jkOLdwWnD..dA66V3tTRzYrmZa2lyqghvsx7Mdaenoaw rP3W3SHMORzN5DdWxqGyvcDoYsV7AW7.1qgoXIn7UMMKTjW5TUmWjE.3hCTH8dr55xueJaB7Wa0m mBZN5WHhDlsXRJrdKpgOCBVRc_SQJhaMd2rM75Luaj2vI3GcSs9alai0bereL2_3AWbJBHmCtOes n.zJBKcck.zuVbiTpXsZ4.0YONK0adQpU3NWte3FZjix7XBJwEn9OE9UcUydTyPqpI8bmXkjHkkY 6HwDTzuji2FEwCHNv66Zsi0wQw6.85Xl8BFvniV93GfPTc5R7pWw8StwRXgCITB3fjXLA29N8iWB DmMTwYWsGKyKjunlPuvPXtJPF.FeJh.SPbzIeHlosf08iQ7bSxXG0Cmkd89sAScqt5QRvRLDeBPV 5XQMj9I4Uv5EcmikNgzmCPujEv29E0fqQS7SYfpyHergx4BZsU68PzIcY5OaRohOVndadsJAmRoW s7QGWBvWz89toOq2MukA1s9.Fj9koGnSJ6hX46Ie74s6RQ6xyRixyFex7b1jnNLQQ5jlPS9tBa8q T8EOXKKCiKL8ggIvrNRgYKrHxCfApFDK1QLR3LkRQdBl2AAQ6AEodCIPzTcnCvFvNbn_1BRrhd1v qlPLSiNQgn0j2vWp9iBNyHJ0l3szcq6l6OeVoXH2su7cthk47IIIIItt6VtuWrceGJdVvv4WapZ5 g2h51O0I3_MoCh0_j4LxRHN6aLxEAIBVB1T2eW.nyXmsZ2Gepg6X3VMjXRluD7VmzkJf0sgLmLWh ozb1M.PRkzHkWbnZMqwNF8dcPlLQLdSjiFLo0HCRue0A3im_I0btN6yqMnhpqEcewJVtmY5i.Jyi IZhymvEYSMWCK8T0ROuhd_DtBwP6ox0yvLFz8x9WvSO5KbfHVYi4o1by4Z0o18A57Pfqga8bRqxq mKI_ZkwVaIaIfb9zrpCnGOpF0YjS7BX8zmYtGKIZLLJKn9K6HMGXihA.vMh2oq7OllSeQTv25JRN dC7K2VrNWiEb7_YrNPs48APm9r3ksCo.vLlcErd28RZ1ZaYYI0Kbufyl1wo9oR0nG0G2QP4bqMT4 08kLdpwvx6O.iw.hBtig3O8ROTnEiXPD4YUHeM5wSSh0UNUrScR5IVUEdU2n9OF4ArzZoWDT0i.6 4yG1sfnv.cXovaIxdmOp2c7BfT1DFhRxyrj0eftVq3tZl4w9xXVVcybp9cc_V2cRwrZdfOSbhetZ VtjJlxy6GlTydkcnfMdIGt.dq5Qe7wRK3uO6E6giCtaavl2m66K7Md3Xw0_MgAkYKBFxmiQrXE85 .VvhNj5698aZlSdCvtSLGJqbkAT3WmWUO.aViqoDz0uA4bMnW_HocG5._84rtyzIFX61XpuZvB0T 0vfMo0_aMy4Vl1RTlpMTt..sJjAeEke7b0_uXCv6ykdwSTFVmxGfCNLQR7noLUgn7_7FXNp8wCLf N145DRtXpgnX8M7Bhj6Q5bGLPMlFHpWbkU8s3m7bW0uueOk5wniUZmgbyzJyg6._b7zF2w4cLchb WToV.4KGxiyA2CMziP3PRm2nHoWuhZMeQqi4aOp8yW082Q_dyv5ZFp.SvIHEbgBev.cjwuuVw9KD t_w5GCBq8cMTL6DhdXi6pcVdzGPAT3.lbI.tSRQJFOcX16YMvdxCo9Ki8uL614q7y445EsHeZ.lb xi98bPnHsVoVgnPU0F2uyzioec0VlvTqCjR7ky3X6.swao_Lsrh8PZ5dC7xsCSGRfjQ.xvp47wbt bGq4n0X44RJoTndY0oufQaQd1duGUtk3L7dIKtI4.A2UPSzw5j.HjKyr9BhJzafv9_uYRkTIFdOB 09w9yVElmavSp02fyZZHXEQqimG4HxeIliLKISc79XET2wI1W.sL9HlF3Or_5mhrhAtR89Vgzpo6 BtQxYpIp5gUY3h5TWkbzugR1w5gAAI8V7b.8QUYTGSPA0r1UgljOwkdYqTjX8AhH8lMV6xzJLt4u ixaGh2OQ1s3Mxstb3RJ895f01NWOfl.izU3gUlRiT366c8I4vt1vBKR4Twbwul1oU7Eh62tBOwIN VM6cu X-Sonic-MF: X-Sonic-ID: 66bf0cad-09c3-49a0-86eb-98b68a75db09 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Mon, 14 Aug 2023 04:04:45 +0000 Received: by hermes--production-ne1-7b767b77cc-27nt8 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 497b70d9c4cde05edcba6ad7f533a9c8; Mon, 14 Aug 2023 04:04:40 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: OpenSSL 3 ports fallout Message-Id: Date: Sun, 13 Aug 2023 21:04:29 -0700 To: Gleb Popov , Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RPLQ80ljgz3JTL Gleb Popov wrote on Date: Sun, 13 Aug 2023 20:30:48 UTC : > Some of the ports I'm using are failing to build after OpenSSL 3 > import due to the following problem. OpenSSL headers that are shipped > in base contain declarations of various deprecated functions for which > libcrypto.so doesn't contain definitions. Some of them are > RSA_generate_key and ERR_* family. These declarations aren't guarded > by any #ifdef and are visible for ports software. VirtualBox and > net-p2p/cardano-node detect these functions, try to use them and then > fail to link due to undefined references. I believe this should be > fixed in the base rather than patching each port? FreeBSD ports is using a 2021-08-24 version of cryptography/hazmat/ for which there have long been more recent versions that no longer have the kind of issue below: File = "/usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/opens= sl/binding.py", line 14, in from cryptography.hazmat.bindings._openssl import ffi, lib ImportError: = /usr/local/lib/python3.9/site-packages/cryptography/hazmat/bindings/_opens= sl.abi3.so: Undefined symbol "ERR_GET_FUNC" =46rom what I wrote on the lists back on 2023-Jul-10: QUOTE In use: security/py-cryptography is at 3.4.8 (2021-08-24) Vintage fixed: 35.0.0 of cryptography dates back to 2021-09-29. Current for cryptography is 41.0.1 (2023-06-01). END QUOTE The full fix is inside cryptography, not in the environments that use it (such as FreeBSD). To me it looks like the tradeoffs now in place suggest adjusting the constraints that are stopping progressing past 3.4.8 in ports, even if there are some other consequences for some people. Note: The above message is from a broken kyua test's backtrace. A bunch of kyua testing involves use of python and ends up with cryptography/hazmat/ involved (and, so, the python is broken in some way). =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Aug 14 14:00:10 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPbd91G4Kz4qXGJ for ; Mon, 14 Aug 2023 14:00:13 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPbd90Z5sz3K3B for ; Mon, 14 Aug 2023 14:00:13 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692021613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fJAPwTomrKzf7EnmPDSWek5AIc/qLOSr4EPqA44vRhQ=; b=OiDwEU1qGdX674Wjt4TiZ655AFPXJr1cYH6rqUhVbnlIWvvUoOwUA10H3jktBjHZI7e9LW wooTKkfjOXCbz4b69PDA1TuOjm4al+LsimCe6wEbWHminixZ6j+s9O8FMF+SE3wRtVTYW8 IX3akRozFj86WbeKe2YJ8UR+uc4fSCMTiS5AM7ZSxod5HhloN13ZxXiP99tO/0rBoH76QM L348pxFtInSKWFW26GLjIIoDvP8LpTT0GN++cARg4XrOktP+S66az/eguTOoIbfHnyYYEp ZN0wI0HsafHUUtDQsxDYjeoWQ9ZSDuO71dshfqEW97Jga561zq5WytWE8BXPzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692021613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fJAPwTomrKzf7EnmPDSWek5AIc/qLOSr4EPqA44vRhQ=; b=THBYDyEI1wpZnMZOxNL65x1TLZWJ2VLd+1zi1BUU9KA9nSxHM1sW4cvCqV673EnoKWWDEt 0V9OXq0l0Aqp2ZCa8C2trZdmoiqu0vm7IG8oONVs4eDw7VejDsTaTV2LUc7bekut1lgsnq 4qerj3wc9Uj0eVCOR63bj40aA9+1+ZzChe4XJOrYw4gMgRJiQKXyl9ItplBa65AkC7oZIf C6BWMfuUYc+Yw7LsVb1e1X8RYvcCwfXgHXlWL/Ei3JoPRmhQt/kZ7LaOd4VBv/B7TNR41D IMJsFU/HxtPl5Uva4cvgifaFSoX6MH/rDP8iXLSr8f6v8MqTdeGM5dbiYh8c0A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692021613; a=rsa-sha256; cv=none; b=PbpX1YBlevAg6zp8KAP1ezKPnKuSUiM9YjV9eFIpTH/lBs3clwIbvLiZ94x/B+SKDss8m/ VhcBYeNeHFaV18FU71oyhZNZNitqLprgbwHsCl2vky2AjUuKc903NwwMH5zoKx/32PWllG IWCJ2HgpJEudq/UEPDo9uSO2Pkkjr+qjmiCnPSNHUHjpEg/hguiOXlm/TjX5saOyvv1TiB TejrrmF2Hyz/8xjzAb+fYmEIcpY9KUAWH/Jwsk73NNIWvDpgQ3iFBbC+0fuJFMgQ3rae9M 5H8e1dQRcO+u5c8itB+6bEQPx5bKuv+KYTnyY22qKJgTMYp+9AHcWd6FhrBgdA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (ti0187a400-1976.bb.online.no [85.166.95.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RPbd85kMtz1MbB for ; Mon, 14 Aug 2023 14:00:12 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id 5AD13180DC; Mon, 14 Aug 2023 16:00:10 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: current@freebsd.org Subject: Re: ZFS deadlock in 14 In-Reply-To: <86h6p4s64h.fsf@ltc.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Sat, 12 Aug 2023 16:11:10 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Mon, 14 Aug 2023 16:00:10 +0200 Message-ID: <86a5utrafp.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dag-Erling Sm=C3=B8rgrav writes: > Trying to narrow this range down, I did not get a deadlock with > 4e8d558c9d1c (10 June) but I did with b7198dcfc039 (16 June) [...] > Perhaps I should try 4e8d558c9d1c again. I managed to geat a deadlock with 4e8d558c9d1c. Its predecessor 5ca7f02946 appears to be working. I'm going to try to come up with a more efficient way to reproduce the deadlock than running poudriere. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Mon Aug 14 22:29:48 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPpx91fQpz4mT1Z for ; Mon, 14 Aug 2023 22:29:49 +0000 (UTC) (envelope-from pstef@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPpx90wHxz4bbt; Mon, 14 Aug 2023 22:29:49 +0000 (UTC) (envelope-from pstef@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692052189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=u90kH29ZNzl3gT4NVCrf277vXM6E/M+qWUZJQI93IM4=; b=XQf1GbZBmHI1EaYKB3k/6v2NHI/vpnclZR8ksfqY9bj25oGau06N1U1LpDaAOt2VDXAkLG p/Ya7kQpAySAsB0o0JuiS2kWS242r3tB+XH0ElYKlnA35a9Sf2/Bujuk31O7nAL4ytE0Co tNmfyhZ40n09MMgxHREzI9xFIL+8gODeUygKS0eSvmrEdC5NGZ3S2cZqvFBKsak7woVsMN sZdh7vTR+dsVbDvF6jfKqOzp78KC4mLt/Awcne3SvLjulC1izLoUiV/480+kawdp1iB6SO U5pVHJFOoWPzxBmNIjkPVN0rqWR+9L4gX9MkIUgwHPmZ5D4VU04Y63zS6v3BNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692052189; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=u90kH29ZNzl3gT4NVCrf277vXM6E/M+qWUZJQI93IM4=; b=O+waFmb8bMfsA+R7H3nDDMfRx4aysfutGp02GImFMYsYnlIC7UQnR1L0gcre74kHlxhhvQ ushWE3ef1ayZS6syOpxGJ353kf5BwLHMUeFy/agMHX5wncy4PiwNfrnezrQUPmoJ/zMrRV bSUn/3iyOzfI8Sf/yK2N5jqF9mXdRU2xj+QcBVKaB0B3McesFQX7ouUtmlG7RVQZr2HN0e x9Czs/3iYmoHOIGsoWuOiAgAMuPrNH67VMcOXtPjX7tChr+9cUdZBklnh6cfi3o7jbLsBv EfJRU6Zce7V7D4J8K5HI26USw51hQ+i0hqYUZ620Snqvyodhv9gYhfj6kjdYGg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692052189; a=rsa-sha256; cv=none; b=fT2WEGfT9PcrgbwoZzuVLOLe2kslnqb2ZyF3HH5fE7ylTrXu83+pyCbeK1Ac4pQjEmXWju k4UX4297YLsXkGKR4CYNYPDdqHLLlw8MGNvB13G7PstzCMp1zl8EbFMt8I2gGH8KDPor+7 lM3Q3P/y6Bk/m8dscPtjA0kc7J4bJYxmLz0/Ndfta4MXS12Ec2UJKv0nkzOLoAr01Nqf7v Zre1tptNV6T8Oq/qE/Tmuc2Ujpnkph8Fi8m2ALPSyLokeFNf2ePhM7U9cVsG8iVHXI+S13 KkXCVCfkpSocXKhWNx9gNIIyoeTB5agKY3tX2Fe2LHFgnYbrF57+XoGbU0ueUA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: by freefall.freebsd.org (Postfix, from userid 1403) id D6A221522; Mon, 14 Aug 2023 22:29:48 +0000 (UTC) Date: Mon, 14 Aug 2023 22:29:48 +0000 From: "Piotr P. Stefaniak" To: Jamie Landeg-Jones Cc: freebsd-current@freebsd.org Subject: Re: ps(1) bugs and problems Message-ID: References: <202307282307.36SN7b7v026284@donotpassgo.dyslexicfish.net> <202308111132.37BBW23A064898@donotpassgo.dyslexicfish.net> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <202308111132.37BBW23A064898@donotpassgo.dyslexicfish.net> On 2023-08-11 12:32:02, Jamie Landeg-Jones wrote: >How about reverting '-d', and adding "-D" for descending, and "-A" for ascending? I don't like that, because it would take three option-letters in total to implement the same function in different variants. The old -d and the new -D'$^' would be the best in that -d would go back to what it was and -D would provide the much needed feature in two variants (possibly more in the future, if needed) while only taking one option-letter. The only problem is that it looks ugly. For the record, just -d'$^' is impossible, because it would break existing command invocations. Piotr From nobody Mon Aug 14 23:45:40 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPrcr41KBz4q3Ww for ; Mon, 14 Aug 2023 23:45:48 +0000 (UTC) (envelope-from grog@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPrcr3YJYz3FMw; Mon, 14 Aug 2023 23:45:48 +0000 (UTC) (envelope-from grog@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692056748; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WTPeQfjBA0uOoPg0bszlmGyqUbulDbnE8mCkaVOl77w=; b=U/rB6dm8ffZE1OaRdIc6jokvvYEDP7RXUR9tEv/JYw51SHy51MxWkLcYp/D787d6drfXzQ FCGJ03hrnCORH56gLVTR8h9DwdqRCqf853jXAoWjfxJcYE8G9RMHNQGTLkhPu/LN0+gNQI VhQ5XlVdU/Whkzyw8/WmAeUp0SzA4womvk/OKJ1jeWjDKJtvKS1jvM3ehHxqU8BMj4v/qw ORCmtnISdV/DQ3Mgu9aPGdCzTPbjKzjo6WX5/TrE1e95IpGlFGqp90EizbXRdjT/xAM51l tysIjh68xiKuE4KAHgz8O34OtLiicekMXFg07T30gZZUe83aPt1G7yW2xlm64Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692056748; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WTPeQfjBA0uOoPg0bszlmGyqUbulDbnE8mCkaVOl77w=; b=ZfFyUvdTXi4kwx2JXjvo9GtDpoJdqysGvoHCPSfci58M8Z5R/R0+UEn6P4u8pkpf1ADuxZ 2zkd4qgBFvxoKsSYLyfnhhpmR7H1tTmZOxwFiwy8/slmF7l3MlKNkSih31yVUyyIXJvXYd 2UahVhvG8p6iDbnmb3sX0cNPyRJlJ3OwF6hVAb/1FzW/hBiY3ng62arYrxaoh6a6XaeGCA PbcihrSTyC+ltNMNVJoZdlgovdSCpH/qZ5iprMXT/y3bloOJ/EyukznbTAh2w3xGcxfYY5 XrOkv55o1sgxF2ZbKwxDsFZOsZXCZYY5dimSEKcZ4Fbgv/o/vA/eR7h6ivmQhg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692056748; a=rsa-sha256; cv=none; b=ubDVFe7okbZ1BQGRPxy9rB2BTqzKUBntJmd9VByw+EWis3ClEMyKXgAmZGHwSsu7LWjYNk rpW9tjQfJl0IHHEf51sVuAEKqY6ugZSqKwO1HfnmRooF3NxbyroN5EnXpz7sdJrhGceHOG qg91l93gpRCaHXV1y+Cm1ljodIRBfqEQ4vbmhqa1ZDvak+2etLcES5JVIB1N9SJ52JlqU/ 8GJU0OIsF+LxDrz9q1V9QN/TshaQU8n2s1EtZ2Ttq7TxkFkPscCusNqKKLjxAtFQkQZVoz VtCWmvNgxjYkwDvzpuoS7UmsxxAFnecdT8Cs76uhGCnEz2wW81eHUW7GI++bSg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from eureka.lemis.com (121-200-11-253.79c80b.mel.nbn.aussiebb.net [121.200.11.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: grog/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RPrcq27cHz4Wl; Mon, 14 Aug 2023 23:45:47 +0000 (UTC) (envelope-from grog@FreeBSD.org) Date: Tue, 15 Aug 2023 09:45:40 +1000 From: Greg 'groggy' Lehey To: Kevin Bowling Cc: current@freebsd.org Subject: Re: Potential show-stopper in em driver? Message-ID: <20230814234540.GG87066@eureka.lemis.com> References: <20230814124703.3875a7ba266cdf079525eb82@dec.sakura.ne.jp> <20230814043746.GE87066@eureka.lemis.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="XjbSsFHOHxvQpKib" Content-Disposition: inline In-Reply-To: Organization: The FreeBSD Project Phone: +61-3-5309-0418 Mobile: +61-490-494-038. Use only as instructed. WWW-Home-Page: https://www.FreeBSD X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 User-Agent: Mutt/1.6.1 (2016-04-27) --XjbSsFHOHxvQpKib Content-Type: text/plain; charset=us-ascii Content-Disposition: inline [moving to current as requested by bz@] On Monday, 14 August 2023 at 10:09:22 -0700, Kevin Bowling wrote: > > I'm able to replicate this on my I217 using iperf3. It happens > quickly with flow control enabled (default) and takes about 15 minutes > of line rate with flow control disabled. I am looking into the scope > of the issue and will commit a fix or enable chicken bits for affected > parts soon. Thanks. Let me know when you have something and I'll test it. I'll reply to the other message later, but things look better without TCO. Greg -- Sent from my desktop computer. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA.php --XjbSsFHOHxvQpKib Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAmTavKQACgkQIubykFB6QiNQkACglQbJ8qP8/+ovNx2TY7S23c2f x4AAniFSm5Mf4eKSutlkUEUGQuvETLYv =HN+K -----END PGP SIGNATURE----- --XjbSsFHOHxvQpKib-- From nobody Tue Aug 15 00:34:12 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RPsj16gyqz4q6lH for ; Tue, 15 Aug 2023 00:34:29 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RPsj13nXbz3JxY for ; Tue, 15 Aug 2023 00:34:29 +0000 (UTC) (envelope-from kevin.bowling@kev009.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ed1-x534.google.com with SMTP id 4fb4d7f45d1cf-525656acf4bso2141764a12.0 for ; Mon, 14 Aug 2023 17:34:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kev009.com; s=google; t=1692059666; x=1692664466; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=9bTv8PR5ToMJJe48ocjlbsMCJM1zQrJPWXSFgKHxnrY=; b=dH4guf/D3qGuxMTJgi+jqyVcwTpsvNQdXvHBI/1d5Im3xvwi9wTFhoPPV8rAdCRf// RoBMtjwd4MuajFWXdni2q+iyHV+bcbrqTFVCEWxiFjP2acnqHLEH3aw4uD4k1DXS9J6U sWNrP+yGaA9HhLeZNXyMGW+6mSrvESe7GVN8A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692059666; x=1692664466; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9bTv8PR5ToMJJe48ocjlbsMCJM1zQrJPWXSFgKHxnrY=; b=VlMLjczu+aPTdzhHTKqLVrO6LszcyRKSHVasGKuLiazWWR7Up6msalc708OXcDpInT RTwbXiY02IV36eQVku7QOQzggfTY0svY8g6ui+NIhYZ0BujpMHieVamQ4lUINqSbXNxV Nsq+wxZAy1ZKhwuN/NOc83DKW/Ms7GUDhHlNThsK95al4lV5VY2bFRBOJtAdoSA9o/Tb Kt5FPqXBb8pru9mhnE6WlSz9VAbT64xwJ7WaZhgYdYenGMe7e+K4egY9QJG6NCUh9B+Z d0aZaKWLLL//u2m4S4baAHN6u2D3jIjj5y+ntW0T0UibD8ztS9uYDUq1mHtxfb7h993M BFIQ== X-Gm-Message-State: AOJu0YyaAfabxled1Hl9FLRDwyuJ6RPtuH+5nYXht95g3A9o+g1d6v/+ jXt4ASwXMibybNdRaGYnQ1VyuVP5ubA9ADzaGfLNzWEf/CbiQwCss8Q= X-Google-Smtp-Source: AGHT+IHpTYJmio48FJqJmyBcoiecizjgE5MKNyvIb2TnZ7Mz4+6/WucGTPPoNdJBsRmSu+odGpA0Lcz8E6OK9Jlh5D4= X-Received: by 2002:a05:6402:151a:b0:523:3754:a4e1 with SMTP id f26-20020a056402151a00b005233754a4e1mr8744020edw.22.1692059665604; Mon, 14 Aug 2023 17:34:25 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <20230814124703.3875a7ba266cdf079525eb82@dec.sakura.ne.jp> <20230814043746.GE87066@eureka.lemis.com> <20230814234540.GG87066@eureka.lemis.com> In-Reply-To: <20230814234540.GG87066@eureka.lemis.com> From: Kevin Bowling Date: Mon, 14 Aug 2023 17:34:12 -0700 Message-ID: Subject: Re: Potential show-stopper in em driver? To: "Greg 'groggy' Lehey" Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4RPsj13nXbz3JxY X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] On Mon, Aug 14, 2023 at 4:45=E2=80=AFPM Greg 'groggy' Lehey wrote: > > [moving to current as requested by bz@] > > On Monday, 14 August 2023 at 10:09:22 -0700, Kevin Bowling wrote: > > > > I'm able to replicate this on my I217 using iperf3. It happens > > quickly with flow control enabled (default) and takes about 15 minutes > > of line rate with flow control disabled. I am looking into the scope > > of the issue and will commit a fix or enable chicken bits for affected > > parts soon. > > Thanks. Let me know when you have something and I'll test it. I went ahead and reverted: 797e480cba8834e584062092c098e60956d28180 I thought I had a handle on the bug a few times but all I've managed to do is make it take a bit longer to stall so far. Whatever is going on is very subtle and may be outside the em_txrx.c code. > I'll reply to the other message later, but things look better without > TCO. I'm able to reproduce on my machine with the same NIC as yours so I no longer need the requested debugging info. > Greg > -- > Sent from my desktop computer. > See complete headers for address and phone numbers. > This message is digitally signed. If your Microsoft mail program > reports problems, please read http://lemis.com/broken-MUA.php From nobody Tue Aug 15 06:16:38 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ1Ht0Nsdz4qXB0 for ; Tue, 15 Aug 2023 06:16:42 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ1Hs0qfhz4MLK for ; Tue, 15 Aug 2023 06:16:40 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.58.23] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qVnM5-0002wH-Sb for freebsd-current@freebsd.org; Tue, 15 Aug 2023 08:16:37 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37F6Gc0L001406 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Tue, 15 Aug 2023 08:16:38 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37F6Gcs3001405 for freebsd-current@freebsd.org; Tue, 15 Aug 2023 08:16:38 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Tue, 15 Aug 2023 08:16:38 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: security/openvpn does not compile in 14-CURRENT w/ poudriere Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.58.23 X-Spamd-Result: default: False [-2.78 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.984]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQ1Hs0qfhz4MLK security/openvpn fails to build with an error message in the log: ... libc.so.7 libcrypto.so.11 libcrypto.so.30 libdl.so.1 liblz4.so.1 liblzo2.so.2 libnv.so.1 libpkcs11-helper.so.1 libssl.so.11 libthr.so.3 /usr/ports/security/openvpn FAILED: either of libssl libcrypto libraries linked multiple times *** Error code 1 The full log is at http://www.unixarea.de/openvpn-2.6.5.log The job uses via make.conf the SSL from the base: ---Begin OPTIONS List--- ===> The following configuration options are available for openvpn-2.6.5: ASYNC_PUSH=off: Enable async-push support DCO=on: Build with Data Channel Offload (ovpn(4)) support DOCS=on: Build and/or install documentation EASYRSA=on: Install security/easy-rsa RSA helper package EXAMPLES=on: Build and/or install examples LZ4=on: LZ4 compression support LZO=on: LZO compression (incompatible with LibreSSL) PKCS11=on: Use security/pkcs11-helper, needs same SSL lib! SMALL=off: Build a smaller executable with fewer features TEST=on: Build and/or run tests UNITTESTS=off: Enable unit tests X509ALTUSERNAME=off: Enable --x509-username-field ===> Use 'make config' to modify these settings ---End OPTIONS List--- --MAKE_ENV-- OPENSSLBASE=/usr OPENSSLDIR=/etc/ssl OPENSSLINC=/usr/include OPENSSLLIB=/usr/lib ... There is a similar, but closed PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254323 What can I do? matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Tue Aug 15 06:35:01 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ1j359bKz4qYqH for ; Tue, 15 Aug 2023 06:35:03 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ1j25cDkz4PFf for ; Tue, 15 Aug 2023 06:35:02 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.58.23] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qVnds-0002rp-Op for freebsd-current@freebsd.org; Tue, 15 Aug 2023 08:35:00 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37F6Z1rd001436 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Tue, 15 Aug 2023 08:35:01 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37F6Z1Yi001435 for freebsd-current@freebsd.org; Tue, 15 Aug 2023 08:35:01 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Tue, 15 Aug 2023 08:35:01 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: www/firefox does not compile in 14-CURRENT w/ poudriere Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.58.23 X-Spamd-Result: default: False [-2.79 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.993]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQ1j25cDkz4PFf The port www/firefox stops to build in congigure phase with: DEBUG: Executing: `/usr/bin/clang -std=gnu99 --target=wasm32-wasi /tmp/conftest._vo3qtm2.c -c` DEBUG: The command returned non-zero exit status 1. DEBUG: Its error output was: DEBUG: | error: unable to create target: 'No available targets are compatible with triple "wasm32-unknown-wasi"' DEBUG: | 1 error generated. ERROR: Failed compiling a simple C source with the wasm C compiler Which compiler should be used? I set CC=clang via the make.conf. Thanks matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Tue Aug 15 07:11:40 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ2YP37xsz4qc1d for ; Tue, 15 Aug 2023 07:13:29 +0000 (UTC) (envelope-from yasu@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ2YP2dy4z4Tc3; Tue, 15 Aug 2023 07:13:29 +0000 (UTC) (envelope-from yasu@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692083609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ew2rYfusYyZ8KmiNX0uWLOdkYZSqqUs22t8LoiWmjY=; b=rK4eaQQGQljDXfkjLERapuYh3RkD4ZkCTQngFMa/WPr94V5kXnjugyM6iEdAY5XNCcqHR5 5mIO3fUB0jPj1+r7erW9Si8rWDYINEEJFr3Y3A9+fS7jCSvNBtLZ6hYgWyoQsWle9iRy8G gWCDxcKwziGUYPq8oVnl2NcIouVsV1ya1bQr+IkU7jceSN0lGrpBym02DM6p23zTmJ2W7a NYI125CbZmwE8Ku3L87gheDUFLoTgjtrUUSfKUy1RPZmvpT/VgPidtpi6MewBHaUOly+kf A1/jofqhUmLBfTIfr+2fnZBFai8aYSs/GPsz4EvsmxMrLB1ITlee/ukXuX4oDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692083609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5ew2rYfusYyZ8KmiNX0uWLOdkYZSqqUs22t8LoiWmjY=; b=lcSguK5o0oMAxga5SrjB3KYcHZB4o+v+2fKEKTKjhigdPuxkdLhP+vzWqbs+9Xu8Bk9B5d 3qPu6IF1YOAOiYjwCrE2Bq6Y/aCrIvoaWEV2gCoLkWRf3VM8dOzc8z+u+BHT0/xnBq+vVh gDuIhnkx2V9Ryfc497yqZIkdneHTVyk/7wRym1DjeBOTa0cEq5qeUa1VbD62+HKnJlI7i7 Dj63E2uhv7q8Fp0GS9VmiNIJnI/bh7p4neQn0lvnRuu1ZIoN17LsbKC172CDv1gGyt4oYz UT3ZBPJCnftXfzH6KKrkj5mIJ83zsN//+fp4frVx658gDuKFy7CUuTLF8MvQww== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692083609; a=rsa-sha256; cv=none; b=gH+Rh4gmKPaReFvZACdjdTyJ7bQdGhXZAHl5kkSM3NA1mE/QWp1EJe7AuPfmnkSEw+Pfvj ud9fHuep6hnuspxrthZnuprpJKlEOPTdRThsa/1fkDMuKL0l7vla8dyZHHsy8uXZA45kVA I7b7GWQTw+YBRziHU7sYX82fIzFzEqucvS1HdNymc+YNUHiVImn22E3jXvnbdVGY80hstn StlSSZ3EyGJ6PW0fT8qb1+CqeDBjfhKLfwrLsG2rHx5z6w9eGBFZ/sF6gu+2n/GWTgFAPY geg7QBd61E5uffFLjSbeS5I2zvpPS9lSxowEuo82uVMkqKtx9Tr2xXeVq2FXfg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from localhost (unknown [IPv6:240b:11:220:fe00::174:11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) (Authenticated sender: yasu/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RQ2YN3Kg9zC7n; Tue, 15 Aug 2023 07:13:28 +0000 (UTC) (envelope-from yasu@FreeBSD.org) Date: Tue, 15 Aug 2023 16:11:40 +0900 (JST) Message-Id: <20230815.161140.2290198053300852479.yasu@FreeBSD.org> To: guru@unixarea.de Cc: freebsd-current@freebsd.org Subject: Re: security/openvpn does not compile in 14-CURRENT w/ poudriere From: Yasuhiro Kimura In-Reply-To: References: X-Mailer: Mew version 6.9 on Emacs 30.0.50 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit From: Matthias Apitz Subject: security/openvpn does not compile in 14-CURRENT w/ poudriere Date: Tue, 15 Aug 2023 08:16:38 +0200 > > security/openvpn fails to build with an error message in the log: > > ... > libc.so.7 > libcrypto.so.11 > libcrypto.so.30 > libdl.so.1 > liblz4.so.1 > liblzo2.so.2 > libnv.so.1 > libpkcs11-helper.so.1 > libssl.so.11 > libthr.so.3 > /usr/ports/security/openvpn FAILED: either of libssl libcrypto libraries linked multiple times > *** Error code 1 > > The full log is at http://www.unixarea.de/openvpn-2.6.5.log > > The job uses via make.conf the SSL from the base: > > ---Begin OPTIONS List--- > ===> The following configuration options are available for openvpn-2.6.5: > ASYNC_PUSH=off: Enable async-push support > DCO=on: Build with Data Channel Offload (ovpn(4)) support > DOCS=on: Build and/or install documentation > EASYRSA=on: Install security/easy-rsa RSA helper package > EXAMPLES=on: Build and/or install examples > LZ4=on: LZ4 compression support > LZO=on: LZO compression (incompatible with LibreSSL) > PKCS11=on: Use security/pkcs11-helper, needs same SSL lib! > SMALL=off: Build a smaller executable with fewer features > TEST=on: Build and/or run tests > UNITTESTS=off: Enable unit tests > X509ALTUSERNAME=off: Enable --x509-username-field > ===> Use 'make config' to modify these settings > ---End OPTIONS List--- > > --MAKE_ENV-- > OPENSSLBASE=/usr OPENSSLDIR=/etc/ssl OPENSSLINC=/usr/include OPENSSLLIB=/usr/lib ... > > There is a similar, but closed PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254323 > > What can I do? > > matthias I tried build of security/openvpn with following conditions. * Host: 14.0-ALPHA1 amd64 main-n264690-54cfeb84846 GENERIC * Poudriere: 3.3.7 * Jail: same as host * Ports tree: d5418d0957ad (main branch) * Default options setting including all dependencies And it finished successfully. Full build log: https://people.freebsd.org/~yasu/poudriere/data/logs/bulk/curamd64-default/2023-08-15_15h42m59s/logs/openvpn-2.6.5.log HTH. --- Yasuhiro Kimura From nobody Tue Aug 15 07:48:57 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ3LS45wyz4qfJg for ; Tue, 15 Aug 2023 07:49:04 +0000 (UTC) (envelope-from grog@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ3LS3dl2z4Xc8; Tue, 15 Aug 2023 07:49:04 +0000 (UTC) (envelope-from grog@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692085744; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3ZZchn4a2ZkKw9b8tddJo3ZndZ7KM3hFCFr04+ilpMo=; b=tKdKj+XMnD78XdxJR11mFaRBXKCYmk2ITKBwfRdZkvL4jsINouQMN4tBbLC9omY05JIvWP Xyy6nOTxCQCmaPsOvoriRF0fFzaGbi++SV+L1uBT+ddY7BWt/EKqJdQSGcexBmE6qPqjVI Ijpw54lOzHww7qaU3Sn5vlzh3tvTXu6QtL1+EeLu9luzwG+5o0cv134JVBZTgZ/zhcDItV GrHEflSMyLLOUDzFhsTQo14eqH+XNl/BHnyZkA3EqhALj5UOk7+jSRxdxEPVX4FaoLTjKI QYdF6V1FyFRYPSgrnfqbO5k5naPsdIQBGm+zyxS7jRRCgO1pM2xFG1L0EnJp7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692085744; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3ZZchn4a2ZkKw9b8tddJo3ZndZ7KM3hFCFr04+ilpMo=; b=TMAZzRWMqikJWiC0+V6kLQaliy2BCDve7LqSczR7fARQX/r0ZRS/DlNx6nlydx235AmTXT sXvl5cbe2vppcJeUusEIf1gluhVChttqW5QhhzDTBOkzPsXBzALtPbVbySEJRjvkO5m9VE AR1tTMp2/D+/a2BgqNZCHZjw0KDeh+Q0Ar0zTusj/An6VnfVRVMbRGccSY7xL1vqsG8oI9 6HFsOAmnnTXwGgFAUh/n+CcPLqBaiCaFKo1arEWBNvf7TMMzoOzR1eg6bCut5B1ESPCvvL 1pB+wYqIRbIhYdsiO5nCRaPkt/qxr6dTZmtJ5AVISkMMJI3DbraaflGWp5L+Hw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692085744; a=rsa-sha256; cv=none; b=uTwjRthr3ljxMRcoN4y4Ad6Tm1NE868tcV2D3gEyiBTqH2oijo2kOQbtCl35PpdY86fPa3 ftxfG/m/A6yIgBZPFXyZOAk3KhV6ipJg3FlICQQfybR+saAgVt1cvrgAZcWiodON9+0Sbu KNgU7BWhAa38pMvGhemU9JbfnyFi6hSxqjvCl1u8ChyqA3C7TLIoVMHxGMjD78utMkgqOL 2hoY4ZvHxqePfCnb0aPE+fDeTVG/cdGmFnFTYSZ0/NoeLcv9810oJlUpyomXO3vuveXqJl gaFF0zVHpVCS2RxaUbOgRKBuWtapZr1Gh1nNysXr/LA39jhIr/GHsKg8Xx69og== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from eureka.lemis.com (121-200-11-253.79c80b.mel.nbn.aussiebb.net [121.200.11.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: grog/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RQ3LR23MZzkDR; Tue, 15 Aug 2023 07:49:02 +0000 (UTC) (envelope-from grog@FreeBSD.org) Date: Tue, 15 Aug 2023 17:48:57 +1000 From: Greg 'groggy' Lehey To: Kevin Bowling Cc: current@freebsd.org Subject: Re: Potential show-stopper in em driver? Message-ID: <20230815074857.GH87066@eureka.lemis.com> References: <20230814124703.3875a7ba266cdf079525eb82@dec.sakura.ne.jp> <20230814043746.GE87066@eureka.lemis.com> <20230814234540.GG87066@eureka.lemis.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EEx6GiKZGZ1wKUra" Content-Disposition: inline In-Reply-To: Organization: The FreeBSD Project Phone: +61-3-5309-0418 Mobile: +61-490-494-038. Use only as instructed. WWW-Home-Page: https://www.FreeBSD X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 User-Agent: Mutt/1.6.1 (2016-04-27) --EEx6GiKZGZ1wKUra Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Monday, 14 August 2023 at 17:34:12 -0700, Kevin Bowling wrote: > On Mon, Aug 14, 2023 at 4:45=E2=80=AFPM Greg 'groggy' Lehey wrote: >> Thanks. Let me know when you have something and I'll test it. > > I went ahead and reverted: 797e480cba8834e584062092c098e60956d28180 Is it that bad? I had the impression that where it worked, it was an advantage. Couldn't you just leave it there disabled, with the option of enabling it along with a warning that it hasn't been tested on all hardware? Greg -- Sent from my desktop computer. See complete headers for address and phone numbers. This message is digitally signed. If your Microsoft mail program reports problems, please read http://lemis.com/broken-MUA.php --EEx6GiKZGZ1wKUra Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iEYEARECAAYFAmTbLekACgkQIubykFB6QiPkUgCgiACy5CWigynk1cJJMIxQ811R TaAAoLBix8Dtdvrsesu0gggzoe3Hul9O =+jD5 -----END PGP SIGNATURE----- --EEx6GiKZGZ1wKUra-- From nobody Tue Aug 15 09:35:09 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ5j66YCbz4TlCl for ; Tue, 15 Aug 2023 09:35:22 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ5j53ZqMz3CBs for ; Tue, 15 Aug 2023 09:35:21 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.58.23] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qVqSM-0002Ql-Oo for freebsd-current@freebsd.org; Tue, 15 Aug 2023 11:35:19 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37F9ZB67009554 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Tue, 15 Aug 2023 11:35:11 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37F9Z90l009553 for freebsd-current@freebsd.org; Tue, 15 Aug 2023 11:35:09 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Tue, 15 Aug 2023 11:35:09 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: Re: security/openvpn does not compile in 14-CURRENT w/ poudriere Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org References: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.58.23 X-Spamd-Result: default: False [-2.53 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.73)[-0.734]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQ5j53ZqMz3CBs El día martes, agosto 15, 2023 a las 08:16:38a. m. +0200, Matthias Apitz escribió: > > security/openvpn fails to build with an error message in the log: > > ... > libc.so.7 > libcrypto.so.11 > libcrypto.so.30 > libdl.so.1 > liblz4.so.1 > liblzo2.so.2 > libnv.so.1 > libpkcs11-helper.so.1 > libssl.so.11 > libthr.so.3 > /usr/ports/security/openvpn FAILED: either of libssl libcrypto libraries linked multiple times > *** Error code 1 > > The full log is at http://www.unixarea.de/openvpn-2.6.5.log > > The job uses via make.conf the SSL from the base: > > ---Begin OPTIONS List--- > ===> The following configuration options are available for openvpn-2.6.5: > ASYNC_PUSH=off: Enable async-push support > DCO=on: Build with Data Channel Offload (ovpn(4)) support > DOCS=on: Build and/or install documentation > EASYRSA=on: Install security/easy-rsa RSA helper package > EXAMPLES=on: Build and/or install examples > LZ4=on: LZ4 compression support > LZO=on: LZO compression (incompatible with LibreSSL) > PKCS11=on: Use security/pkcs11-helper, needs same SSL lib! > SMALL=off: Build a smaller executable with fewer features > TEST=on: Build and/or run tests > UNITTESTS=off: Enable unit tests > X509ALTUSERNAME=off: Enable --x509-username-field > ===> Use 'make config' to modify these settings > ---End OPTIONS List--- > > --MAKE_ENV-- > OPENSSLBASE=/usr OPENSSLDIR=/etc/ssl OPENSSLINC=/usr/include OPENSSLLIB=/usr/lib ... > > There is a similar, but closed PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=254323 > I've unset PKCS11 in the port's option: # cat /usr/local/etc/poudriere.d/140-CURRENT-options/security_openvpn/options # This file is auto-generated by 'make config'. # Options for openvpn-2.6.5 _OPTIONS_READ=openvpn-2.6.5 _FILE_COMPLETE_OPTIONS_LIST=ASYNC_PUSH DCO DOCS EASYRSA EXAMPLES LZ4 LZO PKCS11 SMALL TEST UNITTESTS X509ALTUSERNAME OPTIONS_FILE_UNSET+=ASYNC_PUSH OPTIONS_FILE_SET+=DCO OPTIONS_FILE_SET+=DOCS OPTIONS_FILE_SET+=EASYRSA OPTIONS_FILE_SET+=EXAMPLES OPTIONS_FILE_SET+=LZ4 OPTIONS_FILE_SET+=LZO OPTIONS_FILE_UNSET+=PKCS11 ^^^^^^^^^^^^^^^^^^^^^^^^^^ OPTIONS_FILE_UNSET+=SMALL OPTIONS_FILE_SET+=TEST OPTIONS_FILE_UNSET+=UNITTESTS OPTIONS_FILE_UNSET+=X509ALTUSERNAME With this it builds fine. matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Tue Aug 15 11:48:26 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ8g03wQpz4TvY0 for ; Tue, 15 Aug 2023 11:48:44 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ8fz3TCtz3Nwh for ; Tue, 15 Aug 2023 11:48:43 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=leidinger.net header.s=outgoing-alex header.b="16/Sog3l"; spf=pass (mx1.freebsd.org: domain of Alexander@Leidinger.net designates 2a00:1828:2000:313::1:5 as permitted sender) smtp.mailfrom=Alexander@Leidinger.net; dmarc=pass (policy=quarantine) header.from=leidinger.net Received: from webmail2.leidinger.net (roundcube.Leidinger.net [192.168.1.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: Alexander@Leidinger.net) by outgoing.leidinger.net (Postfix) with ESMTPSA id 439E550E for ; Tue, 15 Aug 2023 13:48:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1692100109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3imW9ZgEnSmT1HzkLXAz8xGLoX0vftYo/9Fv7V1ps8c=; b=16/Sog3lhalWbR6GbJ40Q+sseCShJMsbay7MpepQ+qgWRyurZGEoLy1TG1Kw9vSYQXP7Qa +2hsn6ejz2V6Vji3BgxbWjqhwivGp2vdNT3rp7QU4SUH/9KzNZJY8gkZAPslvdD4I9jAmD CHn6b/nQwohceElMgUwBNE58vu3iKGnZsHxwihZ4BXJMmO5FqjWep68cGdtoZ6JxgVgUyt ZC0dBPdUoryW5NKn/C7k/MjBNHZUnLc645he4sOut4BR7qdOXaDVffp+gxb2NbndRCTani D+xCG0+r4u4axfReYb3m1vLq+V92tl3Ko0YIDiHwNBIhtUP8ocnWOrbRQaqoyg== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Tue, 15 Aug 2023 13:48:26 +0200 From: Alexander Leidinger To: current@freebsd.org Subject: Strange network issues with -current Message-ID: <502489179881909dca19d3b73edc8e8b@Leidinger.net> X-Sender: Alexander@Leidinger.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[leidinger.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; MIME_GOOD(-0.10)[text/plain]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[leidinger.net:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RQ8fz3TCtz3Nwh Hi, since a while I have some strange network issues in some parts of a particular system. A build with src from 2023-07-26 was still working ok. An update to 2023-08-07 broke some parts in a strange way. I tried again with src from 2023-08-11 didn't fix things. What I see is... strange and complex. I have a jail host with about 23 jails. All the jails are sitting on a bridge, and have IPv6 and IPV4 addresses. One jail is a DNS server for a domain which contains all the DNS entries for all the jails on the system (and more). Other jails have mysql (FS socket for mysql nullfs-mounted into other jails for connecting to mysql via the FS socket instead of the network), dovecot IMAP server, postfix SMTP server, a nginx based reverse proxy and 2 different kinds of webmail solutions (old php74 based on the way out on favour for a php81 based one), a wiki and other things. With the old working basesystem I can login into the old webmail system and read mails. With the newer non-working basesystem I still can login, but the auth-credentials are not stored in the backend-session and as such no mail is listed at all, as this requires subsequent connections from php to dovecot. This webmail system is going via the reverse proxy to the webmail-jail which has another nginx configured to connect to the php-fpm backend. With the new webmail system I can login, read mails, and even are writing this email from. The first login to it fails. The second succeeds. It is not behind the reverse proxy (as it is not fully ready yet for access from the outside (DSL with NAT on the DSL-box to the reverse proxy)), but a single nginx with php-fpm backend (instead of 2 nginx + php-fpm as in the old webmail). The wiki behind the reverse proxy is sometimes working, and sometimes not. Sometimes it is providing everything, sometimes parts of the site is missing (e.g. pictures / icons). Sometimes there is simply a blank page, sometimes it gives an error message from the wiki about an unforseen bug... The error messages in the nginx reverse proxy log for all the strange failure cases is "accept4() failed (53: Software caused connection abort)". Sometimes I get "upstream timed out". When it times out in the reverse proxy instead of getting the accept4-errors, I see the same accept4-error message in the nginx inside the wiki or webmail jail instead. I tried to recompile all the components of the wiki and reverse proxy and php81 based webmail, to no avail. The issue persists. Does this ring a bell to someone? Maybe some network or socket or VM based changes in this timeframe which smell like they could be related and maybe good candidates for a backup-test? Any ideas how to drill down with debugging to have a more simple test-case than the complex setup of if_bridge, epair, jails, wiki, php, nginx, ...? Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From nobody Tue Aug 15 12:05:47 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ93V5vhYz4Txtj for ; Tue, 15 Aug 2023 12:06:30 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) client-signature ECDSA (P-256)) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ93V14Ffz3Rgd for ; Tue, 15 Aug 2023 12:06:30 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=leidinger.net header.s=outgoing-alex header.b=eQGLZcVf; spf=pass (mx1.freebsd.org: domain of Alexander@Leidinger.net designates 2a00:1828:2000:313::1:5 as permitted sender) smtp.mailfrom=Alexander@Leidinger.net; dmarc=pass (policy=quarantine) header.from=leidinger.net Received: from webmail2.leidinger.net (roundcube.Leidinger.net [192.168.1.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: Alexander@Leidinger.net) by outgoing.leidinger.net (Postfix) with ESMTPSA id 6FECF312 for ; Tue, 15 Aug 2023 14:05:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1692101183; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=DMlAqUx25Vn984TbSArXQR6fUwgSKJafbJTJOHgGLEE=; b=eQGLZcVfeo4pJSiLUb0m5IY2Xy2c2M59sU31mxAm6DZaFjHEPrw2GTvE+SKy2ojmfCDUyI ir+zSLcljmpr1n7mNeNODPkGdTLLKFJeUF7eTi/jLHDGMbzMMpkxRSaueZFZcb7DZJv9C1 gaW1Mz5oWOd0kdAaQa7N2faJ4zGVgIWXRNKVZ4VVDwGAkxpgV3GgWmxZ8XLTbdQJL8d/yW dr4hea6If89B1lvyT1R/8pLqKL0wl2Q5WczZuGVXAvUrVsFt0FIXIuFem9w9l9MvpJ22zo j/GC0RTcy3q+ubIBngYODDxwKztx0zcJ7gOMjdDITj3AM3tDSCZ4KXc/alc4Gg== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Tue, 15 Aug 2023 14:05:47 +0200 From: Alexander Leidinger To: current@freebsd.org Subject: Speed improvements in ZFS Message-ID: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> X-Sender: Alexander@Leidinger.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[leidinger.net,quarantine]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; R_SPF_ALLOW(-0.20)[+mx:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DKIM_TRACE(0.00)[leidinger.net:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RQ93V14Ffz3Rgd Hi, just a report that I noticed a very high speed improvement in ZFS in -current. Since a looong time (at least since last year), for a jail-host of mine with about >20 jails on it which each runs periodic daily, the periodic daily runs of the jails take from about 3 am to 5pm or longer. I don't remember when this started, and I thought at that time that the problem may be data related. It's the long runs of "find" in one of the periodic daily jobs which takes that long, and the number of jails together with null-mounted basesystem inside the jail and a null-mounted package repository inside each jail the number of files and congruent access to the spining rust with first SSD and now NVME based cache may have reached some tipping point. I have all the periodic daily mails around, so theoretically I may be able to find when this started, but as can be seen in another mail to this mailinglist, the system which has all the periodic mails has some issues which have higher priority for me to track down... Since I updated to a src from 2023-07-20, this is not the case anymore. The data is the same (maybe even a bit more, as I have added 2 more jails since then and the periodic daily runs which run more or less in parallel, are not taking considerably longer). The speed increase with the July-build are in the area of 3-4 hours for 23 parallel periodic daily runs. So instead of finishing the periodic runs around 5pm, they finish already around 1pm/2pm. So whatever was done inside ZFS or VFS or nullfs between 2023-06-19 and 2023-07-20 has given a huge speed improvement. From my memory I would say there is still room for improvement, as I think it may be the case that the periodic daily runs ended in the morning instead of the afteroon, but my memory may be flaky in this regard... Great work to whoever was involved. Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From nobody Tue Aug 15 12:24:15 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ9SB5X1Hz4mJsk for ; Tue, 15 Aug 2023 12:24:26 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) client-signature ECDSA (P-256)) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ9SB13Bgz3VVb for ; Tue, 15 Aug 2023 12:24:26 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=leidinger.net header.s=outgoing-alex header.b=SnX2GaYO; spf=pass (mx1.freebsd.org: domain of Alexander@Leidinger.net designates 2a00:1828:2000:313::1:5 as permitted sender) smtp.mailfrom=Alexander@Leidinger.net; dmarc=pass (policy=quarantine) header.from=leidinger.net Received: from webmail2.leidinger.net (roundcube.Leidinger.net [192.168.1.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: Alexander@Leidinger.net) by outgoing.leidinger.net (Postfix) with ESMTPSA id 9AC9B496 for ; Tue, 15 Aug 2023 14:24:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1692102259; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pKfBThCfycJMnlviNLSuBONqrmdMyADE+CUXWYUrQ7g=; b=SnX2GaYO/VQ2tj7IiobAIq6xlJisErQdIMsu3/tp0UZp1HdFADWSBlpAzePGnPRm/YyE3w FpSytDX6Xa4KMYqSaVgCxQzqQr+MPJghY14JJ/RKAwCvxmvqKh3SHaeq+kT5ExVwH7ETLg Rg8td6wmOqIGbjaKhKiRX7u3w6zKxCGQ94KAUMub9bds8fe6fLOLKmGMieEeqZWkpwIODH EowVq/Po+jD9vp89prVSa54e+X30bnvaWafDDfyhK7YZLulLnfuIPhNQDt+uHJfwDGBGjG dSyGyDeFzSPAO9uwfn+fxGHQYF/+BiLBd7r3V0EsjcR0m718MivSFh0b/pz1tg== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Tue, 15 Aug 2023 14:24:15 +0200 From: Alexander Leidinger To: Current Subject: Re: Strange network issues with -current In-Reply-To: <502489179881909dca19d3b73edc8e8b@Leidinger.net> References: <502489179881909dca19d3b73edc8e8b@Leidinger.net> Message-ID: <395c9565210c69205358bc58bb564900@Leidinger.net> X-Sender: Alexander@Leidinger.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[leidinger.net,quarantine]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; R_SPF_ALLOW(-0.20)[+mx:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DKIM_TRACE(0.00)[leidinger.net:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RQ9SB13Bgz3VVb Am 2023-08-15 13:48, schrieb Alexander Leidinger: > since a while I have some strange network issues in some parts of a > particular system. I just stumbled upon the mail which discusses issues with commit e3ba0d6adde3, and when I look into this I see changes related to the use of SO_REUSEPORT flags, and all my nginx systems use the reuseport directive in their config. I'm compiling right now with his change reverted. Once tested I will report back. Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From nobody Tue Aug 15 12:24:31 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ9SN56p8z4mJyw for ; Tue, 15 Aug 2023 12:24:36 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ9SM25Xkz3VyF for ; Tue, 15 Aug 2023 12:24:35 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=RjswYgkC; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2a00:1450:4864:20::62e as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-99bf91956cdso660250866b.3 for ; Tue, 15 Aug 2023 05:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692102273; x=1692707073; h=in-reply-to:subject:from:references:to:content-language:user-agent :mime-version:date:message-id:from:to:cc:subject:date:message-id :reply-to; bh=j9bvQ2om7HO8rcYvaDCU2f+qEROt2M1Z4Py4jxyOBVQ=; b=RjswYgkC2+NUhv4useEmv8NjAp1bDh8wSuKc9pYZc88BbHYGRivRDaHIPsoZxNMiDo IgfvYds61PtoXgE+N5/rFJKHecYnRbCuDeMsOoMSA5d1IdtfI1HpndxfaX1fVhrxANKr 6dBRlO8SHXYKFaA2Id7Qre6qNms8tozt/piyuu3eKxRHQDngKjM/NwgwSbqIrtwNMwYQ CiOr4dLJtOE9hbHKdzmkh42XXIoLJKLzm3zQrCHAK324UwZAFqZQvRl+vGfBRYrow/1t llCDkCR4TMsA4I/Cp9QWLZzZB6bMmbVexU2D3pVn6tVcDd3V4MRGpXwZmYZUS4Zl1kf6 JVGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692102273; x=1692707073; h=in-reply-to:subject:from:references:to:content-language:user-agent :mime-version:date:message-id:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=j9bvQ2om7HO8rcYvaDCU2f+qEROt2M1Z4Py4jxyOBVQ=; b=TTxksLSbkZEb7pmT+L6EGOC80fZ7i7Oc3KxmPPR0UTDCHzvhtUbtjmx/IwBZeLvHCi s7bPJwYR6GiBDola/d+vANcPqtf0kj+d01bs/QifKsCCW1hPjlxP0HQd/dAkdSCHV3kT q78JRdxxaQwj04QWjLHuTLRutShbdc52Q4oWuNMOVzUm7Bjx1RSLXqQA/KajOiKVr2bF JcXQ0eGzmlY2qmBvD8Ld8HI/l01aolQSDTQ1Aw+DryzMT11DWjgi50TJNWWo5yodsySI p/4/zlrXeHyQyxEfXa7cOtwfHzGUfxLly+2dgb5ZpMxMjWY1DdLoKA/TqBWhBS9lexGG nzig== X-Gm-Message-State: AOJu0YwyXnLAH7NLQ0qe424hjXlsuT2duC+VnsEqLwXry07K0mn07LOd ocmjSZ8HNyxdYbLShIyHwvUCt3pGbL0= X-Google-Smtp-Source: AGHT+IHZOE5WzrlTBRyd3fegmDs7rCwOBhwe75euIS3Vx2FZebt+aGhwG1u3Qc/D1PgKNnqNBKISxw== X-Received: by 2002:a17:906:257:b0:991:b292:699 with SMTP id 23-20020a170906025700b00991b2920699mr8911381ejl.5.1692102272987; Tue, 15 Aug 2023 05:24:32 -0700 (PDT) Received: from [192.168.1.10] (80-42-66-93.dynamic.dsl.as9105.com. [80.42.66.93]) by smtp.gmail.com with ESMTPSA id jo19-20020a170906f6d300b0099bcd1fa5b0sm6956539ejb.192.2023.08.15.05.24.32 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 15 Aug 2023 05:24:32 -0700 (PDT) Content-Type: multipart/alternative; boundary="------------OQ0wPateqPNVxBgvEwMHFGcl" Message-ID: Date: Tue, 15 Aug 2023 13:24:31 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Content-Language: en-US To: freebsd-current@freebsd.org References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> From: Graham Perrin Subject: Re: Speed improvements in ZFS In-Reply-To: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> X-Spamd-Result: default: False [-3.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.977]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[grahamperrin]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62e:from]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RQ9SM25Xkz3VyF This is a multi-part message in MIME format. --------------OQ0wPateqPNVxBgvEwMHFGcl Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 15/08/2023 13:05, Alexander Leidinger wrote: > … periodic runs … Here, I get a sense that these benefit greatly from L2ARC. > … So whatever was done inside ZFS or VFS or nullfs between 2023-06-19 > and 2023-07-20 has given a huge speed improvement. … not within the time frame, the most recent commit was 2023-06-16. the two most recent commits might be of interest. Related: , . --------------OQ0wPateqPNVxBgvEwMHFGcl Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
On 15/08/2023 13:05, Alexander Leidinger wrote:
… periodic runs …

Here, I get a sense that these benefit greatly from L2ARC.

… So whatever was done inside ZFS or VFS or nullfs between 2023-06-19 and 2023-07-20 has given a huge speed improvement. …

<https://cgit.freebsd.org/src/log/?qt=grep&q=L2ARC> not within the time frame, the most recent commit was 2023-06-16.

<https://github.com/search?q=repo%3Afreebsd%2Ffreebsd-src+ZFS+ARC&type=commits&s=committer-date&o=desc> the two most recent commits might be of interest. Related: <https://github.com/openzfs/zfs/pull/14964>, <https://github.com/openzfs/zfs/commit/009d3288dea524c7ad373b04b65bee8bb6f0bfea>.

--------------OQ0wPateqPNVxBgvEwMHFGcl-- From nobody Tue Aug 15 12:28:07 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ9XW6rSkz4mJWm for ; Tue, 15 Aug 2023 12:28:11 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4RQ9XW2Q7bz3XkG; Tue, 15 Aug 2023 12:28:10 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 37FCS7Au063977; Tue, 15 Aug 2023 13:28:08 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 37FCS78i063976; Tue, 15 Aug 2023 13:28:07 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202308151228.37FCS78i063976@donotpassgo.dyslexicfish.net> Date: Tue, 15 Aug 2023 13:28:07 +0100 Organization: Dyslexic Fish To: pstef@freebsd.org, jamie@catflap.org Cc: freebsd-current@freebsd.org Subject: Re: ps(1) bugs and problems References: <202307282307.36SN7b7v026284@donotpassgo.dyslexicfish.net> <202308111132.37BBW23A064898@donotpassgo.dyslexicfish.net> In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Tue, 15 Aug 2023 13:28:08 +0100 (BST) X-Rspamd-Queue-Id: 4RQ9XW2Q7bz3XkG X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] "Piotr P. Stefaniak" wrote: > On 2023-08-11 12:32:02, Jamie Landeg-Jones wrote: > >How about reverting '-d', and adding "-D" for descending, and "-A" for ascending? > > I don't like that, because it would take three option-letters in total > to implement the same function in different variants. Yeah, I can see that. > The old -d and the new -D'$^' would be the best in that -d would go back > to what it was and -D would provide the much needed feature in two > variants (possibly more in the future, if needed) while only taking one > option-letter. The only problem is that it looks ugly. I see why you chose "$" and "^", but wouldn't it look more friendly if you instead used "up" and "down" or "A" and "D" or "forwards" and "backwards", for example? > For the record, just -d'$^' is impossible, because it would break > existing command invocations. Yeah, I can see that. Cheers, Jamie From nobody Tue Aug 15 12:39:56 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ9pC70YDz4mKZR for ; Tue, 15 Aug 2023 12:40:03 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ9pC2NMZz3ZJJ for ; Tue, 15 Aug 2023 12:40:03 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37FCduiU043026; Tue, 15 Aug 2023 21:39:57 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Tue, 15 Aug 2023 21:39:56 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Cc: Matthias Apitz Subject: Re: www/firefox does not compile in 14-CURRENT w/ poudriere Message-Id: <20230815213956.8a0b1ee412b067bf994b8e98@dec.sakura.ne.jp> In-Reply-To: References: Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4RQ9pC2NMZz3ZJJ X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] On Tue, 15 Aug 2023 08:35:01 +0200 Matthias Apitz wrote: > > The port www/firefox stops to build in congigure phase with: > > DEBUG: Executing: `/usr/bin/clang -std=gnu99 --target=wasm32-wasi /tmp/conftest._vo3qtm2.c -c` > DEBUG: The command returned non-zero exit status 1. > DEBUG: Its error output was: > DEBUG: | error: unable to create target: 'No available targets are compatible with triple "wasm32-unknown-wasi"' > DEBUG: | 1 error generated. > ERROR: Failed compiling a simple C source with the wasm C compiler > > Which compiler should be used? I set CC=clang via the make.conf. Maybe this hurts. www/firefox depends on devel/wasi-libcxx, devel/wasi-libc and devel/wasi-compiler-rt13. These are only for llvm/clang13 for now. And clang on main is at 16, but if you don't override the default of www/firefox, it should depend on devel/llvm13 and use it, if base llv/clang is NOT 13. > Thanks > matthias > > > -- > Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 > Public GnuPG key: http://www.unixarea.de/key.pub -- Tomoaki AOKI From nobody Tue Aug 15 12:41:38 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQ9r54PPWz4mLTh for ; Tue, 15 Aug 2023 12:41:41 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc29.google.com (mail-oo1-xc29.google.com [IPv6:2607:f8b0:4864:20::c29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQ9r51GrFz3bsx for ; Tue, 15 Aug 2023 12:41:41 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc29.google.com with SMTP id 006d021491bc7-56c711a889dso3745164eaf.0 for ; Tue, 15 Aug 2023 05:41:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692103299; x=1692708099; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=YTk/uFaz3T0hOAwtba3S3hkj5lmWG02kVEYh1xUMJDE=; b=ayF8Wy954kB9MmYmPQJiqW9+oYcrZYGuCG27uIQTA0LIGHgGTHW5fQwaLr/ARRIKtn erfO+ZmYuPFy4kzpJEhSkW5CKtu4iZ1beLjKBLLt/NH+sPpTZp07kzAICeSRKwIdvO5x NYOPeqkc594+uoCEXVSMazzquGn55xS3B9ZcoYClSSmm9Ox0Oc1d4B52fIaqHdzN2nuZ hEY9aH27yz2ImtlRAIUiJcZ8nJQXwI8G2JdwRzK3htB4e8uOxJUKXowSWjfn6mmlunFX c8Xz7Zpr3sCKXFE0EvKKmuFlAfUPBSIZ5q0wrpdeVZ3nKpcjLzmD0Nr3HoXEyJxGNsuD nISA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692103299; x=1692708099; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YTk/uFaz3T0hOAwtba3S3hkj5lmWG02kVEYh1xUMJDE=; b=ikCMUUfRr+ZGjM/bys8nikVUfBqTESOUQaYoO88j/fG8VCkZjZgbpLnpX3poKY/ZNb wlAfzDKPp6Av7816KCLYBre1XVIQWkgG+fXL49P7J/sQgIMhS7uqeecuSeeTUPRxZcfC rwXm3m/JtQNy34R41P83YNhGeXPdtmhukzCVZbaTHaVGMX8+kdYBkvVVUhdGGILOeig1 xoOYTjDEfZzdii39MhavjTbS6UMqHNDe1qsH92sLglmPBniDQFK8dBKhJEo6/hVlJsjr uCKXaTRf5R3KE+2jETrWrmGTvXieVFz1Og4UBopmlSSZWedSP1LpmdfKgNiHN+2XhvHW ufdQ== X-Gm-Message-State: AOJu0YzVSav1Qz2k7iZZ4F1Nw60fb2iam9PVyzEpZBkC0vjb97L+ZMNN 7e+IWWpT744ZgDRSjKxQ9wtqPFyvdQlht/3ZgGhgJpXvg5c= X-Google-Smtp-Source: AGHT+IFb62cHPIPrip3fvSi2+HalTmoTjy9Hl4ihPCByqHM/zcXiCVqoF8L0JgQEHHsSkAfuhFo/YLzx97B5FIHz400= X-Received: by 2002:a4a:91db:0:b0:566:fba5:e51b with SMTP id e27-20020a4a91db000000b00566fba5e51bmr10594384ooh.7.1692103299438; Tue, 15 Aug 2023 05:41:39 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:745a:0:b0:4f0:1250:dd51 with HTTP; Tue, 15 Aug 2023 05:41:38 -0700 (PDT) In-Reply-To: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> From: Mateusz Guzik Date: Tue, 15 Aug 2023 14:41:38 +0200 Message-ID: Subject: Re: Speed improvements in ZFS To: Alexander Leidinger Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4RQ9r51GrFz3bsx X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 8/15/23, Alexander Leidinger wrote: > Hi, > > just a report that I noticed a very high speed improvement in ZFS in > -current. Since a looong time (at least since last year), for a > jail-host of mine with about >20 jails on it which each runs periodic > daily, the periodic daily runs of the jails take from about 3 am to 5pm > or longer. I don't remember when this started, and I thought at that > time that the problem may be data related. It's the long runs of "find" > in one of the periodic daily jobs which takes that long, and the number > of jails together with null-mounted basesystem inside the jail and a > null-mounted package repository inside each jail the number of files and > congruent access to the spining rust with first SSD and now NVME based > cache may have reached some tipping point. I have all the periodic daily > mails around, so theoretically I may be able to find when this started, > but as can be seen in another mail to this mailinglist, the system which > has all the periodic mails has some issues which have higher priority > for me to track down... > > Since I updated to a src from 2023-07-20, this is not the case anymore. > The data is the same (maybe even a bit more, as I have added 2 more > jails since then and the periodic daily runs which run more or less in > parallel, are not taking considerably longer). The speed increase with > the July-build are in the area of 3-4 hours for 23 parallel periodic > daily runs. So instead of finishing the periodic runs around 5pm, they > finish already around 1pm/2pm. > > So whatever was done inside ZFS or VFS or nullfs between 2023-06-19 and > 2023-07-20 has given a huge speed improvement. From my memory I would > say there is still room for improvement, as I think it may be the case > that the periodic daily runs ended in the morning instead of the > afteroon, but my memory may be flaky in this regard... > > Great work to whoever was involved. > several hours to run periodic is still unusably slow. have you tried figuring out where is the time spent? I don't know what caused the change here, but do know of one major bottleneck which you are almost guaranteed to run into if you inspect all files everywhere -- namely bumping over a vnode limit. In vn_alloc_hard you can find: msleep(&vnlruproc_sig, &vnode_list_mtx, PVFS, "vlruwk", hz); if (atomic_load_long(&numvnodes) + 1 > desiredvnodes && vnlru_read_freevnodes() > 1) vnlru_free_locked(1); that is, the allocating thread will sleep up to 1 second if there are no vnodes up for grabs and then go ahead and allocate one anyway. Going over the numvnodes is partially rate-limited, but in a manner which is not very usable. The entire is mostly borked and in desperate need of a rewrite. With this in mind can you provide: sysctl kern.maxvnodes vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes vfs.recycles_free vfs.recycles Meanwhile if there is tons of recycles, you can damage control by bumping kern.maxvnodes. If this is not the problem you can use dtrace to figure it out. -- Mateusz Guzik From nobody Tue Aug 15 12:48:37 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQB096sQ0z4mLHW for ; Tue, 15 Aug 2023 12:48:41 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQB083VQVz3dCX for ; Tue, 15 Aug 2023 12:48:40 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp; dmarc=none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37FCmbS0044188; Tue, 15 Aug 2023 21:48:37 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Tue, 15 Aug 2023 21:48:37 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Cc: Matthias Apitz Subject: Re: www/firefox does not compile in 14-CURRENT w/ poudriere Message-Id: <20230815214837.063f0655c9ec63575306c69b@dec.sakura.ne.jp> In-Reply-To: <20230815213956.8a0b1ee412b067bf994b8e98@dec.sakura.ne.jp> References: <20230815213956.8a0b1ee412b067bf994b8e98@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-0.49 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; MV_CASE(0.50)[]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.00)[0.003]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCPT_COUNT_TWO(0.00)[2]; HAS_ORG_HEADER(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_NA(0.00)[no SPF record]; DMARC_NA(0.00)[sakura.ne.jp]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Spamd-Bar: / X-Rspamd-Queue-Id: 4RQB083VQVz3dCX On Tue, 15 Aug 2023 21:39:56 +0900 Tomoaki AOKI wrote: > On Tue, 15 Aug 2023 08:35:01 +0200 > Matthias Apitz wrote: > > > > > The port www/firefox stops to build in congigure phase with: > > > > DEBUG: Executing: `/usr/bin/clang -std=gnu99 --target=wasm32-wasi /tmp/conftest._vo3qtm2.c -c` > > DEBUG: The command returned non-zero exit status 1. > > DEBUG: Its error output was: > > DEBUG: | error: unable to create target: 'No available targets are compatible with triple "wasm32-unknown-wasi"' > > DEBUG: | 1 error generated. > > ERROR: Failed compiling a simple C source with the wasm C compiler > > > > Which compiler should be used? I set CC=clang via the make.conf. > > Maybe this hurts. > www/firefox depends on devel/wasi-libcxx, devel/wasi-libc and > devel/wasi-compiler-rt13. These are only for llvm/clang13 for now. > > And clang on main is at 16, but if you don't override the default of > www/firefox, it should depend on devel/llvm13 and use it, if base > llv/clang is NOT 13. Forgot to mention. If you want to force CC=clang for ports other than ww/firefox, maybe below would help. .if !${.CURDIR:M/usr/ports/www/firefox*} CC= clang .endif > > Thanks > > matthias > > > > > > -- > > Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 > > Public GnuPG key: http://www.unixarea.de/key.pub > > > -- > Tomoaki AOKI -- Tomoaki AOKI From nobody Tue Aug 15 13:40:36 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQC8B4S0qz4mQb5 for ; Tue, 15 Aug 2023 13:40:42 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQC8B2bmwz4FR7 for ; Tue, 15 Aug 2023 13:40:42 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; none Received: from [188.174.58.23] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qVuHn-0002Y8-39; Tue, 15 Aug 2023 15:40:39 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37FDebXX026155 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 15 Aug 2023 15:40:37 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37FDealm026154; Tue, 15 Aug 2023 15:40:36 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Tue, 15 Aug 2023 15:40:36 +0200 From: Matthias Apitz To: Tomoaki AOKI Cc: freebsd-current@freebsd.org Subject: Re: www/firefox does not compile in 14-CURRENT w/ poudriere Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: Tomoaki AOKI , freebsd-current@freebsd.org References: <20230815213956.8a0b1ee412b067bf994b8e98@dec.sakura.ne.jp> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20230815213956.8a0b1ee412b067bf994b8e98@dec.sakura.ne.jp> X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.58.23 X-Rspamd-Queue-Id: 4RQC8B2bmwz4FR7 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE] El día martes, agosto 15, 2023 a las 09:39:56p. m. +0900, Tomoaki AOKI escribió: > On Tue, 15 Aug 2023 08:35:01 +0200 > Matthias Apitz wrote: > > > > > The port www/firefox stops to build in congigure phase with: > > > > DEBUG: Executing: `/usr/bin/clang -std=gnu99 --target=wasm32-wasi /tmp/conftest._vo3qtm2.c -c` > > DEBUG: The command returned non-zero exit status 1. > > DEBUG: Its error output was: > > DEBUG: | error: unable to create target: 'No available targets are compatible with triple "wasm32-unknown-wasi"' > > DEBUG: | 1 error generated. > > ERROR: Failed compiling a simple C source with the wasm C compiler > > > > Which compiler should be used? I set CC=clang via the make.conf. > > Maybe this hurts. > www/firefox depends on devel/wasi-libcxx, devel/wasi-libc and > devel/wasi-compiler-rt13. These are only for llvm/clang13 for now. > > And clang on main is at 16, but if you don't override the default of > www/firefox, it should depend on devel/llvm13 and use it, if base > llv/clang is NOT 13. Yes, I deleted CC=clang... from the poudriere's make.conf and it picked-up llvm13 and compiled fine. Thanks anyway for having replied. matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Tue Aug 15 16:04:42 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQGLP07C8z4q4hW for ; Tue, 15 Aug 2023 16:04:45 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQGLN6gGHz4RQH for ; Tue, 15 Aug 2023 16:04:44 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692115484; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YBr0aubhbdJGIpP2Azyqc7rfARZ1OUuac9Ek9/AupRY=; b=rMrKmShbgAj+7nqfH+bOPZi26jKvSfPmamLY1rpNyzJTnWQJphou0Y7ITMTzdJjslN9wf0 L/RYkEOoyxCYkLCwJ0u+0/f4gpmjEMuxICas0TcPpf8VulG77LwhvWtlyDcZYHtjFnTYmm jsJl7qepNvbD4s7iPqgL7lNCk0T21ThBrkA0g56hwz/9el24G5Pa2eP23533jFQ8xvYVta 4Xvr1z4IP4ZEf43LI0CYssX4u0JIPZiaAsczQ7obhcxZ3xVgTriMo5+XsKOT0Db9HPN39P C6bTzWTg9SpgGtcIzgoGpbrL6Cua0MgdbONfI3/uLeWx+icmZzRqh1CzdKTYkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692115484; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YBr0aubhbdJGIpP2Azyqc7rfARZ1OUuac9Ek9/AupRY=; b=mRI/JJLReKC16B679MfJppdgfpy/Ot2V9PQE0VYzrK/HrOVdUwlJnFfaTakrTCHnLTSWtZ jAsfXQcS9e/O1HB3jq46UYQetgskgce+jhiITCaaf2n5wnyfXxS1IkKn8JUBJ90IG+c8Gr fKRlX8FrjJX8ZHzAdp8eTxK0S9tq4YW7g2LXSqM3wXlUnEEEWS5Rx/wSKvhnOKGIG2GiV3 5gvN/qKn3qsge1i1pofAs97pXaYDVThfqFM0dmbD3tw3Ig7kjOE5ke0psx/EnnH3iDaCKZ fSvoqj3gYYyRRj8qBk3nejz9Tm9jdBqGpJmnir5WTBI+SU6ijCCUcXeokp8VFQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692115484; a=rsa-sha256; cv=none; b=ZK8xoTIHE3Ly0NEjRGODXzpFmM7+faZ/cTshfSCcDvaK0YtSclpCVjPSDhU554i+avAb/F PfGyIDlv/9NuEChmguUbC66laXvXjHeIF5GAEnRHl3jZepp96pCLJqW6NI9mvjBOsVe0IA eOvAQrcPpI8KB3QBdHNDt5NcFa+4O16kDEyzv5nLJz6LEnj01dJxY4Obxwv14TzhjGJkt5 Xqow7+faoBd/D6fFCeM6TVqI1a6v23VeURr6wMQzcPasjXAhrv2UiC5lNEMXG7EdUOgm4q iuzUS49hrTa2BhiMbkF1hkLS/EoBURCLeXHAHBHNj375JPmXUHOyo8uSWqJKXA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (unknown [IPv6:2001:4647:d671:0:36e8:94ff:feca:9834]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RQGLN4v6MztQB for ; Tue, 15 Aug 2023 16:04:44 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id 5302418469; Tue, 15 Aug 2023 18:04:42 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: current@freebsd.org Subject: Re: ZFS deadlock in 14 In-Reply-To: <86a5utrafp.fsf@ltc.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Mon, 14 Aug 2023 16:00:10 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Tue, 15 Aug 2023 18:04:42 +0200 Message-ID: <86350kqokl.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dag-Erling Sm=C3=B8rgrav writes: > I managed to geat a deadlock with 4e8d558c9d1c. Its predecessor > 5ca7f02946 appears to be working. I'm going to try to come up with a > more efficient way to reproduce the deadlock than running poudriere. I wrote a script that creates multiple filesystems, snapshots them, populates them and rolls them back continuously but so far I have not succeeded in triggering the deadlock without poudriere. I guess my script doesn't consume enough vnodes. Also, 9228ac3a69c4 (9 August, last commit before the contrib/googletest breakage) still deadlocks. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Tue Aug 15 16:21:41 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQGk03g2fz4q6RD for ; Tue, 15 Aug 2023 16:21:44 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc2c.google.com (mail-oo1-xc2c.google.com [IPv6:2607:f8b0:4864:20::c2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQGk01Fzcz4Tp8; Tue, 15 Aug 2023 16:21:44 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc2c.google.com with SMTP id 006d021491bc7-56d6879dcaaso4061979eaf.3; Tue, 15 Aug 2023 09:21:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692116502; x=1692721302; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=wvTRrgE6qE0O3dV7oIPRQ3Jy+jKoMqKaQAVe/lf08ao=; b=CnCxzkjwGfkqb4aL5Vg5EvP72IToxkFq52HkwtcwJ8Bt8hquBg+Ycl/W63qKxz+BRX AbgqdJCM+O8dTPyQHWHdr44k8Y7DMUjBpvHRuEKqj+QsZJnk9+EoVUCk0LLX0PrvN3PN Gu47/tQebyOSthren/InGpyEPOjjUa4bnXBT92BZT2uyGMn7O765kisu2QTpeCHdREOs VForMqcL336OhaWCSlDfWOhHwxTeHibJBXlcOEDi4QgpYZUeItJws22NboAowu107IRi gDUtFCFvQXuOA7Rkqu8KSsA5WQWBDu98bV+mu64ML2sQ6snXd28GAajdPupmfQIDloUN ssng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692116502; x=1692721302; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wvTRrgE6qE0O3dV7oIPRQ3Jy+jKoMqKaQAVe/lf08ao=; b=cLDVZ9GiE5klQOtoxjlKspO56Py0C4FwAVdCl4DHNCFs/ZbBMzhtcmcAtHHfE93LLF btX3/pbEjKuzqWWGnkLwgGNSIX7BiFvrTHYaZ46rExmunTcHYikGpCIwY3Szz/ShTYyT q7tLwvjBGmCZNj/8i74QFY3jloAeGgUvtYc4/4UtFtJAxIOUaqv7LF32sXx7+ElQaz4Y ahvOVGylhRoccPnBN9Yq3VYrBk5BrFUXLQjTCs4OaQyvfRE5w+c+bhBdxTptxIrfQ5t/ JjZuWiT0ZeaZd28ORG7g63PKfCOnqIoLLhmx64vjAOzwB9Rb21zpS78xdx0F6bD4uHbF j1XA== X-Gm-Message-State: AOJu0YxR0+AiitcWQBnrJokW6YmuJuJYnkvFDrJzVji06VGP7RBbJ5vn ffjVzhhGWCqjCStc/GOE4LYTQZXuMmeBDL1yzUkXzU7cAHQ= X-Google-Smtp-Source: AGHT+IEf3yieWi8wEMpcDyTFYGs4sy4tAI1FLMwXTUm+Uh4xR59GvM+bTaSWTW0Wn3nJZrUeB3U+WbYL5npSS3C/mjU= X-Received: by 2002:a4a:9d57:0:b0:56e:14b7:b053 with SMTP id f23-20020a4a9d57000000b0056e14b7b053mr7452470ook.7.1692116502360; Tue, 15 Aug 2023 09:21:42 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:745a:0:b0:4f0:1250:dd51 with HTTP; Tue, 15 Aug 2023 09:21:41 -0700 (PDT) In-Reply-To: <86350kqokl.fsf@ltc.des.no> References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> From: Mateusz Guzik Date: Tue, 15 Aug 2023 18:21:41 +0200 Message-ID: Subject: Re: ZFS deadlock in 14 To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4RQGk01Fzcz4Tp8 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 8/15/23, Dag-Erling Sm=C3=B8rgrav wrote: > Dag-Erling Sm=C3=B8rgrav writes: >> I managed to geat a deadlock with 4e8d558c9d1c. Its predecessor >> 5ca7f02946 appears to be working. I'm going to try to come up with a >> more efficient way to reproduce the deadlock than running poudriere. > > I wrote a script that creates multiple filesystems, snapshots them, > populates them and rolls them back continuously but so far I have not > succeeded in triggering the deadlock without poudriere. I guess my > script doesn't consume enough vnodes. > > Also, 9228ac3a69c4 (9 August, last commit before the contrib/googletest > breakage) still deadlocks. > Given that the custom reproducer failed I think the most prudent course of action is to reproduce again with poudriere, but this time arrange to have all stacktraces dumped. this should do it: sbin/ddb/ddb.conf:script kdb.enter.panic=3Dtextdump set; capture on; run lockinfo; show pcpu; bt; ps; alltrace; capture off; textdump dump; reset it is a slightly finicky beast so I would trigger a panic by hand first to validate it works as expected. --=20 Mateusz Guzik From nobody Tue Aug 15 16:22:54 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQGlN4mRHz4q6XN for ; Tue, 15 Aug 2023 16:22:56 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQGlN3pc0z4Vp8; Tue, 15 Aug 2023 16:22:56 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692116576; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HM44CCxwx4z5MVgIGQglipxGNJ0ctVlTjJ3ofhv5zt0=; b=RNo9S/S4IH4eRYTxHC9xm32W/WbKQGbhyoxDgi4N9t+mctUwPy/EOIz8IyAofTIdXo94fL FZRHB4DuVE7I6hyIfWixvP+vlhyTQYMR1gOAKxSazPVcSBVbSim8zr/E2W4yj/PihH/rI4 woI7u37Cu6ScSk7iEPSKmBOKfX1FjaChUsCHkq4MnUIGyptuqgKLhmguyflxHtl03vPygN HwEXe5w+oNEY8Feq8tw/cxubqqHvDWHrzw0G+QE/fGF/XARDTc0AEEr1ICsdXzJoiqz6jJ tBuQeFVU3BvGBU4BjnmA0s+vIqhwhpfdktJ/gsDwGFfovU6zFqVTlING096sPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692116576; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HM44CCxwx4z5MVgIGQglipxGNJ0ctVlTjJ3ofhv5zt0=; b=ReSB0db27NXPcal9NfnfH7blB4gEzF7lYdMST0mBNmVlTIeLcskcsPVOuOBuk7Ivz7a3Cc 8W6Q6++MyN4NkGTVJhS+2bk4dn9ovihPqq5mFz2lLTmLi4CCQ9gTQLQZnrJZyaGMr3mMDZ 3ZCxRS19rzluTeQnmFZAh+g8Q8/akr51FVt6NEIkmHnB/8LVlq112aHzLyEhgKVKM2qkqF KZ2jNEx0th+TwjJ5epdtfoOv8sV4ibCAVVHQ/5ahvak2QdL+Mcmo2PlKv5oisf0NSrysv1 r6UcqXZLqthoUi5KAcSTW/I1PffH7/VgyVzYrwyRvdlok/1xuOsjO0p8nyDHGw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692116576; a=rsa-sha256; cv=none; b=OB/rYQUhMkcztyp4WEhEjjrmuVa92mfPoMnAGe5qs5VNBVtEGFhORgywm+V8vu2MnIVTM6 XUYGkAgcvuat9aPuKcSwqdtS2bD6pps20ZW8kjDaXOpC9HLCj4sODM5C9Pb5MtSwyTj9Gk 4BIDTTgz3mhUreh8O7OO6q+PwTj4ZiSuFfDiBjGJ2a4g32Eve8GO5gBtE4uIHb5urP9Y9P eBhPxYLk69FF4ZbLsx/kqfaPaE0Pk3cisvs6tAfc2h8JB8YB3k42hEpI8M7+NGgmHpI96k S2TnWkxL8HkNX4cz11qPpD1vDUL3fMKSfF41KlGnQ2WA+y+w+3etGXBhVx1rXA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (unknown [IPv6:2001:4647:d671:0:36e8:94ff:feca:9834]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RQGlN26cWzs5g; Tue, 15 Aug 2023 16:22:56 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id D20D118171; Tue, 15 Aug 2023 18:22:54 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mateusz Guzik Cc: current@freebsd.org Subject: Re: ZFS deadlock in 14 In-Reply-To: (Mateusz Guzik's message of "Tue, 15 Aug 2023 18:21:41 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Tue, 15 Aug 2023 18:22:54 +0200 Message-ID: <86y1icp95t.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mateusz Guzik writes: > Given that the custom reproducer failed I think the most prudent > course of action is to reproduce again with poudriere, but this time > arrange to have all stacktraces dumped. Why? What more information do you need? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Tue Aug 15 16:26:19 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQGqJ60Vbz4q6vw for ; Tue, 15 Aug 2023 16:26:20 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc2e.google.com (mail-oo1-xc2e.google.com [IPv6:2607:f8b0:4864:20::c2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQGqJ3zGsz4X3G; Tue, 15 Aug 2023 16:26:20 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc2e.google.com with SMTP id 006d021491bc7-56d6dfa8b52so3877764eaf.3; Tue, 15 Aug 2023 09:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692116779; x=1692721579; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=6gOs2Ul666Gfp6aOQ2AF13jY6Bg4mR2vDPAa746aL10=; b=UoAf+a13+ag8pDHx97ZhPoe2DsvDM6zN6aT7no1UeaiL85xZMNw8qNLIMEVtpjXl7c ssMnjWfwcvtNNResZQCv2hjtdPQbRpMtm4MhZSaJT8dhBfoyytR1bCauFI85795rvN82 9on7AACh+vLtAT45BJXEKOXntZNDc1ABW+N4nbwFGSuPEjjrKAqH0p4ajs71Op3i3YO8 gFOgLBUBiLMuqWzb/3se308hIgDl0WBUH2dPupNYLCIglWF5W5AHNnpX1ZUaMHJ7JaFN nB4AB8s93H0oH8gkV4r175C2AdK2We9DRAmnEwnVfXCNF24obCjl7d0IeBHyD6Judv3U zbuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692116779; x=1692721579; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6gOs2Ul666Gfp6aOQ2AF13jY6Bg4mR2vDPAa746aL10=; b=Il6gOj/bO2gHPGcgtBPSgIpxuQ6WVOJ3zxy69GwgubJqsv5wrrBDjkUT9yOuJ94/1d 0PCVlYN85lITTK6f3VA36/iofpX/z0SVuG+qxbxs1mRxKiWo9nH9aZ5S7+aOu4A6ea1i JIgQpc2KNZACO+xaRuT9sle0FVingUpKGvz+YJEHnrgFVzx2+fbL9y5LdCrzHAocEKH8 3NuI/rH6evb93nu7teg5aF9ogAQcLgGkrCJWFQTjtpkkSliZgvLoeh1Y0SJUcR6Korvs U/7mWiw7gmJDOfaSbYUtyN/LAoiNefO5X4zhJqSz5qrECKsfE0ctjmexMIECeoUvXg0L y7Aw== X-Gm-Message-State: AOJu0YwNlyKDCMIsXmGbCovpv2nGlY5zU/AEOv8q54o0CcBqxCOKaBU5 +v6Wv9ecYpjlw+m8EpZvEgPcsKbXOGX/fwInqb+lXR1wnrE= X-Google-Smtp-Source: AGHT+IHUsiiKpLYgiX+v417ugU4yYFwwabyCQ/PEhAFNKL+mtKRNGHKZQf+7KFUUwkpd+fWSDLydQdXp0oc9TIivbqA= X-Received: by 2002:a05:6870:61cc:b0:1ad:e92:62e1 with SMTP id b12-20020a05687061cc00b001ad0e9262e1mr13447873oah.54.1692116779641; Tue, 15 Aug 2023 09:26:19 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:745a:0:b0:4f0:1250:dd51 with HTTP; Tue, 15 Aug 2023 09:26:19 -0700 (PDT) In-Reply-To: <86y1icp95t.fsf@ltc.des.no> References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> From: Mateusz Guzik Date: Tue, 15 Aug 2023 18:26:19 +0200 Message-ID: Subject: Re: ZFS deadlock in 14 To: =?UTF-8?Q?Dag=2DErling_Sm=C3=B8rgrav?= Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4RQGqJ3zGsz4X3G X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 8/15/23, Dag-Erling Sm=C3=B8rgrav wrote: > Mateusz Guzik writes: >> Given that the custom reproducer failed I think the most prudent >> course of action is to reproduce again with poudriere, but this time >> arrange to have all stacktraces dumped. > > Why? What more information do you need? > Going through the list may or may not reveal other threads doing something in the area and it very well may be they are deadlocked, which then results in other processes hanging on them. Just like in your case the process reported as hung is a random victim and whatever the real culprit is deeper. --=20 Mateusz Guzik From nobody Tue Aug 15 16:28:16 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQGsc0HhTz4q6gK for ; Tue, 15 Aug 2023 16:28:20 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQGsb6hWTz4YC3; Tue, 15 Aug 2023 16:28:19 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692116899; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jRtDWPpIi6kkPs0rwWJezA33BWxKmdWsbe9eRz+wSmA=; b=f6ChVdcUAy72vRcIGP/SmfKFQcAnThrm9pgr8/EapeRMK4LyL8j5LkfXxLiAE/6b/Ct3wn NdRSH+t8iHxtWGl93DxRvidoNZbpOC63B+3OwNGrePljV/94gzfYyMmTbiy3qFTZKdVP0B pfpGSIyWPbTe46d9R/ehYTnKon//FZ/F0L71DhPbC5muj2YheMIEsc9JoD2ROzTzOdLc/i HHxddLes9FFrmX1WPainhTsa3LjTwQo0VGdc9AUzl1b5cLFqgtVEDAes8H8elN9YvyjqKQ gt9qojelakE33z6zwvfLRpIFsBFo680Rc5U+1bAq9kHSFUVberpHeTooEaeewA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692116899; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jRtDWPpIi6kkPs0rwWJezA33BWxKmdWsbe9eRz+wSmA=; b=Ips0oCGZVjH/rUg6R8njlcMCizIgdol6kuKqh4gZMjXuuJ35dYvIvKv69+aZMJU60KFOg2 1RyoZaxxvu4kBk5LbqbC0/O3RPa/tNdkCrlXWoZq1ZIPqwo0duYClwmtXw3LJX/C+oWN0r Iau3q91dOPNwFLws6JyOUu0PC83wFKNR+VpVMr4/UBnWCCaNxUCsEXRZECNf3FRzT42Xiq 9/izjZZHF18B3llkiONn/Ttv3mN4wFEPF6NzgvK7k7VvMTnbh0IskjLu5yxpcFV+5U6A0S IIETb1l2R8YI/cY/M/qEzwtiOOcEbS8fIlqAhBNA/cRXoMwEW/MONZf7lnfVQA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692116899; a=rsa-sha256; cv=none; b=YyrGi+VtE3uSENlQ01KsnY5fb90bbr9bGyz5r4jbC8oj92gmQ2diZ8YzuBnn+/jYEnEIAZ qrizSqZKcj9/vfP7PGQ+/4eDDE3EpJyLar4Y+CZQ8E/eKl15NJH1OF96RGAiBXRvyT4s61 FbqMWuEYZBbUs7IAhII+khdJQqIzkEqWbgW8p1H0Jj89fX//kYYmSpyxLMT75vGkZRD/Fx pi+l9aLQodSiV9ip5V0EwH2iWn1ciAdDFa3Xhe7+iUZ2jJuQHOhvcBC4ozvE4hLk++ErVR zHG8N5wSwgXM1egAxSTYzzfJOyjdZyHJWHTnbSZQGo47uw2jipufr9LjaI3rwQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (ti0187a400-1976.bb.online.no [85.166.95.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RQGsb3s53zv9n; Tue, 15 Aug 2023 16:28:19 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id 009AC18172; Tue, 15 Aug 2023 18:28:16 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Mateusz Guzik Cc: current@freebsd.org Subject: Re: ZFS deadlock in 14 In-Reply-To: (Mateusz Guzik's message of "Tue, 15 Aug 2023 18:26:19 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Tue, 15 Aug 2023 18:28:16 +0200 Message-ID: <86ttt0p8wv.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mateusz Guzik writes: > Going through the list may or may not reveal other threads doing > something in the area and it very well may be they are deadlocked, > which then results in other processes hanging on them. > > Just like in your case the process reported as hung is a random victim > and whatever the real culprit is deeper. We already know the real culprit, see upthread. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Tue Aug 15 17:37:45 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQJPm2Bq6z4qC4X for ; Tue, 15 Aug 2023 17:37:48 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQJPm1gqjz4fFD for ; Tue, 15 Aug 2023 17:37:48 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692121068; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QHegJcAr2ziZ2AQnStzY+ZlioBqBsjbYRT52c4bpuPc=; b=AckrAoV2vnPCecmzIpGHulyzRlf/3Vl3UhVs/4RkRVxWsZXYrTlWUy23MnaRSmCj0Ju6JD YhyKIu+w0VXQPrpZ7x+k8uIt3TP9hE8oyrVT+4+5Of5bDdm9CYf0sMwwMBRrPA7O4gJbNm rDTSFbVgYdLEBKp9LqJNytECiJGdXFEJzJ3F2Myk9/3X1NNy7/b2w1gtsX3AHkh0Jvvxri Bam4LBr0w/V1jPKNa0cDhByq3UWaMnpBgXFtbYkvs7wwEWdHlznWzyi6dF+Di7C9/SKdzN PewbwBtHSmbu/7D2nkrOrXfOs7NrdPSIvtk1RAschecc4im7waVNtLdx0JI9mA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692121068; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QHegJcAr2ziZ2AQnStzY+ZlioBqBsjbYRT52c4bpuPc=; b=nUSictF3uLjSZSGfVlWIMFKsMli/e9wrm1bUIrqSAlIKgdJ/0xFre2vG4rvUzjo959ICL6 3XoXsfDQkz85NgZIWaAhjvWNHgch5790fUuQylbWRxqgkVpfm9PEBM/FGFztMLappuz5W+ Ov7euxLRRHEn6TnpVQigorJq7NGOCaHlJu/qdd+omoR6HbPXXx8xtkb5QUrGcVMNKpY/1/ GIM05eABJgsxAEVVkbBuJm2YRKTtyFeprfBMrS66i/rqhFFe1+AjQtUCyZRMHSrxfUWeK0 jfBnkXEgV+IVmz9I3mg7bZ/QdguXxPJuX44zK8DOf9MONvQCyZzw2Up8yahBnA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692121068; a=rsa-sha256; cv=none; b=Mnndu0FYXQUzljS/QETOdSNJ9BKJuBB9sw4dar73/noCfBhEc4hpXUZnr0KOn4Xn8sOTlU 8iRdtolkTemVtIefZQt3/G5gcqHLb94lRVhtTknmcBS+iLCvJtyN9ugf1edZmuCHvJJ7p6 AxF+Tam7cDYRPR1FNdSxuPbkRK14PC8GhMUxRg0HgHcVrt2IRKM1XWjrCqVfaZDqPq1CP0 T+5dC19obOOI0rJOFp9RnpPdgcw6zWsz+bdEbmRRyhBACMjW31ZWSMh1OqHJARsIyw3Jlh AidIuVmz4+qhT8zynImq2wMR13etZRS986EBVFsQMizMhLB+3KUDyb9advyL5Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (ti0187a400-1976.bb.online.no [85.166.95.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RQJPl6vwgzw4R for ; Tue, 15 Aug 2023 17:37:47 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id 3616318DC7; Tue, 15 Aug 2023 19:37:45 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: current@freebsd.org Subject: Re: ZFS deadlock in 14 In-Reply-To: <86350kqokl.fsf@ltc.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Tue, 15 Aug 2023 18:04:42 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Tue, 15 Aug 2023 19:37:45 +0200 Message-ID: <86pm3op5p2.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable The attached script successfully deadlocks 9228ac3a69c4. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=zfs-deadlock.sh #!/bin/sh : ${n:=$(nproc)} : ${pool:=zroot} basefs="${pool}/zfsdl" set -eu zfs destroy -r "${basefs}" >/dev/null 2>&1 || true zfs create -o com.sun:auto-snapshot=false "${basefs}" basedir="$(zfs get -H -o value mountpoint "${basefs}")" echo "preparing tarball..." >&2 tarball="${basedir}/zfsdl.tar" mkdir "${basedir}/src" (cd /usr/src ; find * -type d) | (cd "${basedir}/src" ; xargs mkdir -p) (cd /usr/src ; find * -type f) | (cd "${basedir}/src" ; xargs touch) tar cf "${tarball}" -C "${basedir}" src zfs_deadlock() { local fs=$1 dir zfs create "${fs}" dir="$(zfs get -H -o value mountpoint "${fs}")" zfs snapshot "${fs}@empty" while ! [ -f "${basedir}/stop" ] ; do echo "fill ${fs}..." >&2 tar xf "${tarball}" -C "${dir}" echo "rollback ${fs}..." >&2 zfs rollback -rR "${fs}@empty" done } for i in $(seq -w "${n}") ; do zfs_deadlock "${basefs}/${i}" & sleep 1 done wait echo "stop" >&2 zfs destroy -r "${basefs}" --=-=-=-- From nobody Tue Aug 15 19:24:35 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQLnv2dVfz4qK0P for ; Tue, 15 Aug 2023 19:25:23 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQLnn40Fwz3Rjv for ; Tue, 15 Aug 2023 19:25:17 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=leidinger.net header.s=outgoing-alex header.b=J4tqb5Pf; spf=pass (mx1.freebsd.org: domain of Alexander@Leidinger.net designates 2a00:1828:2000:313::1:5 as permitted sender) smtp.mailfrom=Alexander@Leidinger.net; dmarc=pass (policy=quarantine) header.from=leidinger.net Received: from webmail2.leidinger.net (roundcube.Leidinger.net [192.168.1.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: Alexander@Leidinger.net) by outgoing.leidinger.net (Postfix) with ESMTPSA id 22D74304 for ; Tue, 15 Aug 2023 21:24:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1692127509; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zx1/NAg+l9+mbS5zgFVZPCuGIg9czQHLQjzu+wPHmCY=; b=J4tqb5PflOY7ci38nn6v0aK7xvQVsBQ+gw//dBbRFr1slu1qIdnhR6GbLyFXxdJKYAJiNA H2VNtUq5DFYSeWSZYDaQ3Xu9qpym4CbqRnWNOgpA1jwBlIUZYAHgg4yxspEUbsf9w7PzjV a148bHHt5OEbc5dmQo4ld/36lB5xn9ZL2C2RN72/Rwr2r4DRursF7iNemBxQYoVrjEjbBH BDq/Ith4NGyAdSbLJcQtUVfcWdmYdJcU/AlPnvDdV1wNxNz+eXDNqs8CazXVrFgtViEu3C b6rU3mcxJXGQE5qdSBEtJqaNaJCL9eF+UHKO+7P7Qj7Bs1RuXUQaTTSvFDdREw== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Tue, 15 Aug 2023 21:24:35 +0200 From: Alexander Leidinger To: Current Subject: Re: Strange network issues with -current In-Reply-To: <395c9565210c69205358bc58bb564900@Leidinger.net> References: <502489179881909dca19d3b73edc8e8b@Leidinger.net> <395c9565210c69205358bc58bb564900@Leidinger.net> Message-ID: X-Sender: Alexander@Leidinger.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[leidinger.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[current@freebsd.org]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE]; BLOCKLISTDE_FAIL(0.00)[2a00:1828:2000:313::1:5:query timed out]; FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[leidinger.net:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RQLnn40Fwz3Rjv Am 2023-08-15 14:24, schrieb Alexander Leidinger: > Am 2023-08-15 13:48, schrieb Alexander Leidinger: > >> since a while I have some strange network issues in some parts of a >> particular system. > > I just stumbled upon the mail which discusses issues with commit > e3ba0d6adde3, and when I look into this I see changes related to the > use of SO_REUSEPORT flags, and all my nginx systems use the reuseport > directive in their config. I'm compiling right now with his change > reverted. Once tested I will report back. Unfortunately it wasn't that. Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From nobody Tue Aug 15 19:33:34 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQLzX0H0fz4qKZt for ; Tue, 15 Aug 2023 19:33:44 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) client-signature ECDSA (P-256)) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQLzW4pzwz3TM1 for ; Tue, 15 Aug 2023 19:33:43 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; none Received: from webmail2.leidinger.net (roundcube.Leidinger.net [192.168.1.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: Alexander@Leidinger.net) by outgoing.leidinger.net (Postfix) with ESMTPSA id 1F3C43BA; Tue, 15 Aug 2023 21:33:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1692128018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=no1F18xUql/JGhp1P337LBi4Ya41Ca8CQbUOlgkg2i8=; b=qCjkZm9lXeFvkAZ57zvyrqkH0+quu01Nr6KrwX5MxkT5gweMxvpo87hzoD0JJB3iQTb5xQ Nv14+BYYYUzAuzB3t5P/4fQvdKVJFZmNAj/NUZWLu1kaME8BjQmj4unLwgRP7T9Yo6YzuV nrogebh9s4IN6sU+f+e8cb5e0M+M6plMZw2erMumjfpUbe87k7sDntBwo0dPSFIIVvp7Sv 34ZhUELcxjEZ8BiLKnJ9lFSqBamWGxUFo1SVrgxJdPBC89RQm4IMNbCWPcDzGrl1Y8G1rD kNlo+CDxcNDjQViJpBc+QpIc2Vz+NnRBAw4Jarv3RdZ4a1DkXEaVJkV9jhH1sA== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Tue, 15 Aug 2023 21:33:34 +0200 From: Alexander Leidinger To: Mateusz Guzik Cc: current@freebsd.org Subject: Re: Speed improvements in ZFS In-Reply-To: References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> Message-ID: X-Sender: Alexander@Leidinger.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4RQLzW4pzwz3TM1 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE] Am 2023-08-15 14:41, schrieb Mateusz Guzik: > With this in mind can you provide: sysctl kern.maxvnodes > vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes > vfs.recycles_free vfs.recycles After a reboot: kern.maxvnodes: 10485760 vfs.wantfreevnodes: 2621440 vfs.freevnodes: 24696 vfs.vnodes_created: 1658162 vfs.numvnodes: 173937 vfs.recycles_free: 0 vfs.recycles: 0 > Meanwhile if there is tons of recycles, you can damage control by > bumping kern.maxvnodes. Looks like there are not much free directly after the reboot. I will check the values tomorrow after the periodic run again and maybe increase by 10 or 100 so see if it makes a difference. > If this is not the problem you can use dtrace to figure it out. dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or something else? Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From nobody Tue Aug 15 21:03:17 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQNz82MvQz4qQJZ for ; Tue, 15 Aug 2023 21:03:32 +0000 (UTC) (envelope-from joe@via.net) Received: from smtp3.via.net (smtp3.via.net [157.22.3.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "via.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQNz800W7z3cl9 for ; Tue, 15 Aug 2023 21:03:31 +0000 (UTC) (envelope-from joe@via.net) Authentication-Results: mx1.freebsd.org; none Received: from mail.via.net (mail.via.net [157.22.3.34]) by smtp3.via.net (8.15.2/8.14.1-VIANET) with ESMTPS id 37FL3TWb025687 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=FAIL); Tue, 15 Aug 2023 14:03:30 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.1 at smtp3.via.net Received: from smtpclient.apple ([209.81.2.65]) by mail.via.net (8.15.2/8.14.1-VIANET) with ESMTP id 37FL3RRf017532; Tue, 15 Aug 2023 14:03:27 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.1 at mail.via.net From: joe mcguckin Message-Id: <20BA3146-9448-4B34-9B2A-ED591664B68F@via.net> Content-Type: multipart/alternative; boundary="Apple-Mail=_4232FA49-C650-4C57-B92E-09FD76B590F5" List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\)) Subject: Re: Speed improvements in ZFS Date: Tue, 15 Aug 2023 14:03:17 -0700 In-Reply-To: Cc: Mateusz Guzik , current@freebsd.org To: Alexander Leidinger References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> X-Mailer: Apple Mail (2.3696.120.41.1.1) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (smtp3.via.net [157.22.3.7]); Tue, 15 Aug 2023 14:03:30 -0700 (PDT) X-Rspamd-Queue-Id: 4RQNz800W7z3cl9 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US] --Apple-Mail=_4232FA49-C650-4C57-B92E-09FD76B590F5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii When I watch the activity leds of the disks on one of our ZFS servers, I = notice there will be a burst of activity for 3 or 4 seconds, then no = activity for a couple of seconds, then it repeats. Is that normal? Thanks, joe Joe McGuckin ViaNet Communications joe@via.net 650-207-0372 cell 650-213-1302 office 650-969-2124 fax > On Aug 15, 2023, at 12:33 PM, Alexander Leidinger = wrote: >=20 > Am 2023-08-15 14:41, schrieb Mateusz Guzik: >=20 >> With this in mind can you provide: sysctl kern.maxvnodes >> vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes >> vfs.recycles_free vfs.recycles >=20 > After a reboot: > kern.maxvnodes: 10485760 > vfs.wantfreevnodes: 2621440 > vfs.freevnodes: 24696 > vfs.vnodes_created: 1658162 > vfs.numvnodes: 173937 > vfs.recycles_free: 0 > vfs.recycles: 0 >=20 >> Meanwhile if there is tons of recycles, you can damage control by >> bumping kern.maxvnodes. >=20 > Looks like there are not much free directly after the reboot. I will = check the values tomorrow after the periodic run again and maybe = increase by 10 or 100 so see if it makes a difference. >=20 >> If this is not the problem you can use dtrace to figure it out. >=20 > dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or = something else? >=20 > Bye, > Alexander. >=20 > --=20 > http://www.Leidinger.net Alexander@Leidinger.net: PGP = 0x8F31830F9F2772BF > http://www.FreeBSD.org netchild@FreeBSD.org : PGP = 0x8F31830F9F2772BF --Apple-Mail=_4232FA49-C650-4C57-B92E-09FD76B590F5 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii When = I watch the activity leds of the disks on one of our ZFS servers, I = notice there will be a burst of activity for 3 or 4 seconds, then no = activity for a couple of seconds, then it repeats.
Is = that normal?

Thanks,

joe


Joe = McGuckin
ViaNet Communications

joe@via.net
650-207-0372 cell
650-213-1302 = office
650-969-2124 fax

On Aug 15, 2023, at 12:33 PM, Alexander Leidinger <Alexander@Leidinger.net> wrote:

Am = 2023-08-15 14:41, schrieb Mateusz Guzik:

With this in mind can = you provide: sysctl kern.maxvnodes
vfs.wantfreevnodes = vfs.freevnodes vfs.vnodes_created vfs.numvnodes
vfs.recycles_free vfs.recycles

After a reboot:
kern.maxvnodes: 10485760
vfs.wantfreevnodes: 2621440
vfs.freevnodes: = 24696
vfs.vnodes_created: 1658162
vfs.numvnodes: 173937
vfs.recycles_free: 0
vfs.recycles: 0

Meanwhile if there is tons of recycles, you can = damage control by
bumping kern.maxvnodes.

Looks like there are not much = free directly after the reboot. I will check the values tomorrow after = the periodic run again and maybe increase by 10 or 100 so see if it = makes a difference.

If this is not the problem you can use dtrace to figure it = out.

dtrace-count on = vnlru_read_freevnodes() and vnlru_free_locked()? Or something else?

Bye,
Alexander.

--
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org    netchild@FreeBSD.org =  : PGP 0x8F31830F9F2772BF

= --Apple-Mail=_4232FA49-C650-4C57-B92E-09FD76B590F5-- From nobody Tue Aug 15 21:18:37 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQPJr2N6nz4qQxW for ; Tue, 15 Aug 2023 21:18:52 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQPJq2Bb8z3fWL for ; Tue, 15 Aug 2023 21:18:51 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com; dmarc=none Received: by mail-lj1-f173.google.com with SMTP id 38308e7fff4ca-2b9b50be31aso88220351fa.3 for ; Tue, 15 Aug 2023 14:18:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692134329; x=1692739129; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=M4AIEHrmCy22RtuIPjkV9NS9j9fPlw9sTna3dIAuPso=; b=dq88wSJTLo/i1aDymNFwoDdtpnPXYGdNuXAJXKaeDn+j48XAjubP+Rt0d8YaM/sq7P 4tNUetbIguQqPeFzx0Y/t8KnNRKQXb9a+oYE3jgdAq5kRkx8f8om4IC1QVLlX7wgsiHB sKW3WLcYqnFjHPPdfOVBtlfhZ1C6r+QzHf8n7aMSb63PJuSJJonQZJenSy9ctpv5D9Ol pvXqpcNH4NQThmgoLthptKvg5hS1zfqiBs9qcQu1CvfnIgsweAwDm0uP6SEbu6QBXYxV sby7riNr0T4WffRyV1jqy7WFz9A6DmcL176ZX+Hu01nhQdIIDY/56XU8b3Dk3lleygDb CbJg== X-Gm-Message-State: AOJu0Yz3aUBOyCpY0kCWZAaudWPmP3eDfVOMXOYKfPHBhzU2DI2UHDL4 lOg4EXOwnm700NgIa0zhVl4ioNNU3pvBdWmmOzFyszunI5A= X-Google-Smtp-Source: AGHT+IHKbW5TfzRKJXbgmxsHmZnJDfkJSMtBM7CsdpowBSNRYEEr4bWES14ON6caK+Ko2/nyv4Vn5zr4o/X94CpZ3rY= X-Received: by 2002:a2e:a174:0:b0:2b6:da1e:d063 with SMTP id u20-20020a2ea174000000b002b6da1ed063mr9442825ljl.45.1692134328593; Tue, 15 Aug 2023 14:18:48 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 From: Ed Maste Date: Tue, 15 Aug 2023 17:18:37 -0400 Message-ID: Subject: Defaulting serial communication to 115200 bps for FreeBSD 14 To: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-2.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.992]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[209.85.208.173:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.208.173:from]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[carpeddiem]; ARC_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQPJq2Bb8z3fWL FreeBSD currently uses 9600 bps as the default for serial communication -- in the boot loader, kernel serial console, /etc/ttys, and so on. This was consistent with most equipment in the 90s, when these defaults were established. Today 115200 bps seems to be much more common, and I'm proposing that we make it the default for FreeBSD 14.0. I have a review open: https://reviews.freebsd.org/D36295. There are a few minor nits in the review to be addressed still but assuming there's general agreement I'll iterate on those and commit this in a few logical chunks. From nobody Tue Aug 15 21:29:45 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQPYR21Yqz4qRmv for ; Tue, 15 Aug 2023 21:29:47 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oa1-x29.google.com (mail-oa1-x29.google.com [IPv6:2001:4860:4864:20::29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQPYQ4yCFz4Cn0 for ; Tue, 15 Aug 2023 21:29:46 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oa1-x29.google.com with SMTP id 586e51a60fabf-1c4c5375329so2404018fac.2 for ; Tue, 15 Aug 2023 14:29:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692134985; x=1692739785; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Wcp/Wyu7R77A5GAps1LUbVJYfFVXP407Nudom8aSM0o=; b=cL7YERv2+10yMuSHBE/lFsAzw1YpNDLw3bvsKXm/wyLts00fJo+0iP/GzBYpx5Poxr NnHAxjPBuOrZiKmhdXSluXEKFQm5+uOnFBEqIcoRmNhqVoVypqhVqH1LKG0zGdC0T0jV cY1WeWarjWuxp/SJioPd4RGUcbc5bb6xoHAyATT5ChBXpHc4Y5fuJr/n2Mw3yn0tWlok I1Dt8QKUT32n3AUGCfPwJTeSo4xgMLbgbxP11MJS50/+XuyUtZxcMHHUwq4yNXFkDUAw LurMfpqHM+qSs/flhyFVrSjDQkW8GbngF/FHhK1QFLgVhVGH3OqpQPBlo/Vwnfsh1nP0 1pZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692134985; x=1692739785; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Wcp/Wyu7R77A5GAps1LUbVJYfFVXP407Nudom8aSM0o=; b=VwhMZplbzMU3mBI2vjNEPbEOMpNWQoBc504nwQIsSw0D76xaEYWXhMXgcpmgVYz5vd VO2AncAmqpc3lyXWP0GwQ0PMjPrXKT5rRcuVS5zsIKN78tXcEQpzDjnLRR217GYekuH8 d+hWuNtTfw8VgZoLBbC9dUFxW7CBjWDtQP6TGMCFboA6FlVOH2A0MhIuOlza1zl98n+8 gQgFUuonGhz1uqUjXxqCQNUlp+o+qQBEJtbkN5h4SyK9VboQ4MY556Ngc1XF7B6UiYX5 l+xAqbCXdpftB718andtk3WALFqw7vVFBSOjw8Ojp1lx+U+i4bW2BqIC4GpAS8h+p8fb dEsA== X-Gm-Message-State: AOJu0YxNKpwaxOhugenL5Gscb7oW5Nh9iur2TqKOi87Z4nbaZBTjgxdp UA5kDf50HO2HsT3eqI/YBoMajjZX4gpGlB3L0NkB07HP5zQ= X-Google-Smtp-Source: AGHT+IHakXZUHNnSL1llhO9+XrlTJQm8T8eHlsGf6QQaERcuDwvP+z3vK09hKLCH/q58DJpgzTPnebY/D9uDSC1HZUE= X-Received: by 2002:a05:6870:e990:b0:1bf:ce5b:47a with SMTP id r16-20020a056870e99000b001bfce5b047amr13746451oao.52.1692134985598; Tue, 15 Aug 2023 14:29:45 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:745a:0:b0:4f0:1250:dd51 with HTTP; Tue, 15 Aug 2023 14:29:45 -0700 (PDT) In-Reply-To: References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> From: Mateusz Guzik Date: Tue, 15 Aug 2023 23:29:45 +0200 Message-ID: Subject: Re: Speed improvements in ZFS To: Alexander Leidinger Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4RQPYQ4yCFz4Cn0 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US] On 8/15/23, Alexander Leidinger wrote: > Am 2023-08-15 14:41, schrieb Mateusz Guzik: > >> With this in mind can you provide: sysctl kern.maxvnodes >> vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes >> vfs.recycles_free vfs.recycles > > After a reboot: > kern.maxvnodes: 10485760 > vfs.wantfreevnodes: 2621440 > vfs.freevnodes: 24696 > vfs.vnodes_created: 1658162 > vfs.numvnodes: 173937 > vfs.recycles_free: 0 > vfs.recycles: 0 > >> Meanwhile if there is tons of recycles, you can damage control by >> bumping kern.maxvnodes. > > Looks like there are not much free directly after the reboot. I will > check the values tomorrow after the periodic run again and maybe > increase by 10 or 100 so see if it makes a difference. > >> If this is not the problem you can use dtrace to figure it out. > > dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or > something else? > I mean checking where find is spending time instead of speculating. There is no productized way to do it so to speak, but the following crapper should be good enough: #pragma D option dynvarsize=32m profile:::profile-997 /execname == "find"/ { @oncpu[stack(), "oncpu"] = count(); } /* * The p_flag & 0x4 test filters out kernel threads. */ sched:::off-cpu /execname == "find"/ { self->ts = timestamp; } sched:::on-cpu /self->ts/ { @offcpu[stack(30), "offcpu"] = sum(timestamp - self->ts); self->ts = 0; } dtrace:::END { normalize(@offcpu, 1000000); printa("%k\n%s\n%@d\n\n", @offcpu); printa("%k\n%s\n%@d\n\n", @oncpu); } just leave it running as: dtrace -s script.d -o output kill it after periodic finishes. it blindly assumes there will be no other processes named "find" messing around. > Bye, > Alexander. > > -- > http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF > http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF > -- Mateusz Guzik From nobody Wed Aug 16 02:44:29 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQXXs23hTz4q4bS for ; Wed, 16 Aug 2023 02:44:45 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-qv1-f51.google.com (mail-qv1-f51.google.com [209.85.219.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQXXr3nCTz4hRT for ; Wed, 16 Aug 2023 02:44:44 +0000 (UTC) (envelope-from ctuffli@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 209.85.219.51 as permitted sender) smtp.mailfrom=ctuffli@gmail.com; dmarc=none Received: by mail-qv1-f51.google.com with SMTP id 6a1803df08f44-6473fefcd23so4864666d6.3 for ; Tue, 15 Aug 2023 19:44:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692153883; x=1692758683; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ELhnwMgQ5gveIq39i4lgyeyQptll9eriYAKvMeHiAfw=; b=YarMbJ9I9mxfjrQLBXzyXSVhJVo5a+9K48Abr5wckJoOKm1/6RNVfX+M0aQUML73Xv IWOXdYTrB6Y2mcVpGs0LaKogHCwtUDO6hrktKZ09NI1SLdNdE4uAS+/8Ls8Y5dsWyUoz 68eybfobh7ZA1jrASm/MXMmlED8Tm/l2HKQNG03mQZHv6jdHoRFhDJAJet1CSxJSgcJm dLFy72aHTXT4xfA0gUBZvBW3r12MIw0cxn3C9j6Ki71/fFSiqxF+Vo8SAINIRi9nXN2e d6Av+aXLATSssllK2Au8w4sYekfF3nnxkVQG5oDzrghAcgNPX/q246FBWZe60dwb1hJ8 A6WQ== X-Gm-Message-State: AOJu0YzOwsuD3LXU032CkjEXjOETeA3eEIPVAH/V6xAOpw44TE5etMjb 91wV+3YLYV+YxjYIlRJbfSs55lifOAQ= X-Google-Smtp-Source: AGHT+IEbKStgTfHzN5yUF3oUPthLzYIVDeO2icsFvZHfgDHefEA5TazS3HP+bpOwnvfA/K3AN+Q7Ow== X-Received: by 2002:a0c:f345:0:b0:647:395c:8df9 with SMTP id e5-20020a0cf345000000b00647395c8df9mr498658qvm.46.1692153883038; Tue, 15 Aug 2023 19:44:43 -0700 (PDT) Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com. [209.85.160.179]) by smtp.gmail.com with ESMTPSA id f5-20020a0ccc85000000b0063d252a141dsm4536002qvl.116.2023.08.15.19.44.42 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 15 Aug 2023 19:44:42 -0700 (PDT) Received: by mail-qt1-f179.google.com with SMTP id d75a77b69052e-40ff82320a7so36933211cf.3 for ; Tue, 15 Aug 2023 19:44:42 -0700 (PDT) X-Received: by 2002:a05:622a:1105:b0:403:fee4:2e7f with SMTP id e5-20020a05622a110500b00403fee42e7fmr854542qty.36.1692153882482; Tue, 15 Aug 2023 19:44:42 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Chuck Tuffli Date: Tue, 15 Aug 2023 19:44:29 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Can't assign address to igc To: FreeBSD-Current Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.63 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.97)[-0.974]; NEURAL_HAM_SHORT(-0.66)[-0.659]; FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[209.85.219.51:from,209.85.160.179:received]; FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; R_DKIM_NA(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.219.51:from]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQXXr3nCTz4hRT [Adding freebsd-net@] On Tue, Aug 1, 2023 at 10:46=E2=80=AFAM Chuck Tuffli wr= ote: > > Running a recent-ish version (n264266-8f8da1bcc799) on an Intel NUC > (RNUC11PABi5), assigning an IPv4 address to igc0 doesn't work. There > is no error message, and this has been working in the past. Looking > through dmesg only shows: > > igc0: link state changed to UP > igc0: link state changed to DOWN > igc0: link state changed to UP > > The address is set in rc.conf: > ifconfig_igc0=3D"inet 192.168.5.10 netmask 255.255.255.0" > > And manually setting it via > # ifconfig igc0 inet 192.168.5.10/24 up > does not work either. Any suggestions? I spent a little time debugging this but still need some help understanding what might be wrong. Instrumenting dump_sa() in the AF_INET case, the .sin_addr contains the value of the IP address I'm adding (printf shows 0x0a05a8c0 or big endian 192 168 6 10). Switching the log level in sys/netlink/route/iface.c to LOG_DEBUG3 shows messages: [nl_iface] dump_iface_addr: dumping ifa 0xfffff8000db0cd80 type inet(2) for interface igc0 Does this mean the address information was written to the net link buffer? If so, what might make ifconfig not display that address? One other tidbit is, eventually ifconfig reports the address and things like sshd start working. Typically this takes < 100 iterations to do this. From nobody Wed Aug 16 02:46:21 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQXb24XFcz4q4Nf; Wed, 16 Aug 2023 02:46:38 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-qt1-f181.google.com (mail-qt1-f181.google.com [209.85.160.181]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQXb120YNz4jNf; Wed, 16 Aug 2023 02:46:37 +0000 (UTC) (envelope-from ctuffli@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 209.85.160.181 as permitted sender) smtp.mailfrom=ctuffli@gmail.com; dmarc=none Received: by mail-qt1-f181.google.com with SMTP id d75a77b69052e-40fd6da0325so36471281cf.3; Tue, 15 Aug 2023 19:46:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692153995; x=1692758795; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/C8ArVJPnTcSxhqoTBF26Ii7i/A6X6F2VznlSi/1wQI=; b=jR1bfVWot/c7NK12+YNW3CKcKeobF95kP6LdY/TNnbLOJghZXGU9brRZfLI4RDPlLi VTb6vUO/MUNVf2ZX4+l0avEIdvdObEqHivt6tvIZbknsgyboptufL+oYThXgn4WXW+5p Giyq+VtCEcuArhUJLYE6Z+NQWDvBDAj9nio7Y2CCZSWLeEDO+EremzbxdQcCsIjkluGs QEHxrShnObVxpPFCWRQo15Q/ueYzJ4hsWxxmbvVZ8lM5y/ukDr4GfpC7PErivEUnkW3S UwsGcrr1hWHArq/YnLo5bzPf0IiVWptgTxuKScT45HfAbId0ToQKMLWRzbHO/NI/Ww3e e+TQ== X-Gm-Message-State: AOJu0Yzuxg5Thor22BVQC3dw8sCQG3OhZhRdpg12ADCUxVQK0G+Wzb6S w48yClMclDn/w7bL1z9QQo1xs0WRLUo= X-Google-Smtp-Source: AGHT+IGBzkS8VucktJF0m4cI9eutsTE0/esgZXehDYUaiXXZvetfbDbfVPwsaOOJORBORkAKn/wZeA== X-Received: by 2002:ac8:5716:0:b0:403:fdf7:2e90 with SMTP id 22-20020ac85716000000b00403fdf72e90mr821869qtw.29.1692153995295; Tue, 15 Aug 2023 19:46:35 -0700 (PDT) Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com. [209.85.219.48]) by smtp.gmail.com with ESMTPSA id ff10-20020a05622a4d8a00b0040ff0590fd2sm4230797qtb.87.2023.08.15.19.46.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 15 Aug 2023 19:46:35 -0700 (PDT) Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-640c5df2e6eso30709386d6.1; Tue, 15 Aug 2023 19:46:34 -0700 (PDT) X-Received: by 2002:ac8:5904:0:b0:40f:db31:3a8e with SMTP id 4-20020ac85904000000b0040fdb313a8emr606537qty.46.1692153994696; Tue, 15 Aug 2023 19:46:34 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Chuck Tuffli Date: Tue, 15 Aug 2023 19:46:21 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Can't assign address to igc To: FreeBSD-Current , freebsd-net@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.74 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.64)[-0.645]; FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; RWL_MAILSPIKE_GOOD(-0.10)[209.85.160.181:from]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.160.181:from,209.85.219.48:received]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org,freebsd-net@freebsd.org]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[freebsd.org]; FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQXb120YNz4jNf Bah, actually adding freebsd-net this time On Tue, Aug 15, 2023 at 7:44=E2=80=AFPM Chuck Tuffli wr= ote: > > [Adding freebsd-net@] > > On Tue, Aug 1, 2023 at 10:46=E2=80=AFAM Chuck Tuffli = wrote: > > > > Running a recent-ish version (n264266-8f8da1bcc799) on an Intel NUC > > (RNUC11PABi5), assigning an IPv4 address to igc0 doesn't work. There > > is no error message, and this has been working in the past. Looking > > through dmesg only shows: > > > > igc0: link state changed to UP > > igc0: link state changed to DOWN > > igc0: link state changed to UP > > > > The address is set in rc.conf: > > ifconfig_igc0=3D"inet 192.168.5.10 netmask 255.255.255.0" > > > > And manually setting it via > > # ifconfig igc0 inet 192.168.5.10/24 up > > does not work either. Any suggestions? > > I spent a little time debugging this but still need some help > understanding what might be wrong. > > Instrumenting dump_sa() in the AF_INET case, the .sin_addr contains > the value of the IP address I'm adding (printf shows 0x0a05a8c0 or big > endian 192 168 6 10). Switching the log level in > sys/netlink/route/iface.c to LOG_DEBUG3 shows messages: > [nl_iface] dump_iface_addr: dumping ifa 0xfffff8000db0cd80 type > inet(2) for interface igc0 > Does this mean the address information was written to the net link > buffer? If so, what might make ifconfig not display that address? > > One other tidbit is, eventually ifconfig reports the address and > things like sshd start working. Typically this takes < 100 iterations > to do this. From nobody Wed Aug 16 04:02:57 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQZH81MMcz4q8qg for ; Wed, 16 Aug 2023 04:03:00 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta002.cacentral1.a.cloudfilter.net (omta002.cacentral1.a.cloudfilter.net [3.97.99.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQZH80H3Yz4rCK; Wed, 16 Aug 2023 04:03:00 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; none Received: from shw-obgw-4004a.ext.cloudfilter.net ([10.228.9.227]) by cmsmtp with ESMTP id VzG6qyGCf6NwhW7kJqVNn8; Wed, 16 Aug 2023 04:02:59 +0000 Received: from spqr.komquats.com ([70.66.152.170]) by cmsmtp with ESMTPA id W7kHqBmyp3fOSW7kIqzsRr; Wed, 16 Aug 2023 04:02:59 +0000 X-Authority-Analysis: v=2.4 cv=J8G5USrS c=1 sm=1 tr=0 ts=64dc4a73 a=y8EK/9tc/U6QY+pUhnbtgQ==:117 a=y8EK/9tc/U6QY+pUhnbtgQ==:17 a=kj9zAlcOel0A:10 a=UttIx32zK-AA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=EkcXrb_YAAAA:8 a=b191eoINUNPb834Z3dEA:9 a=/c6rEPcUitil17zVHlIbtIAEuH0=:19 a=CjuIK1q_8ugA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id AE30144D; Tue, 15 Aug 2023 21:02:57 -0700 (PDT) Received: by slippy.cwsent.com (Postfix, from userid 1000) id A397366; Tue, 15 Aug 2023 21:02:57 -0700 (PDT) X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.8+dev Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Ed Maste cc: FreeBSD Current Subject: Re: Defaulting serial communication to 115200 bps for FreeBSD 14 In-reply-to: References: Comments: In-reply-to Ed Maste List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <4199.1692158577.1@slippy> Date: Tue, 15 Aug 2023 21:02:57 -0700 Message-Id: <20230816040257.A397366@slippy.cwsent.com> X-CMAE-Envelope: MS4xfMAEpfDBUxDeQDT8jyyhzTtNmSvXlCxKFYQ6JMHd1gHs6lu0bHk2v/tehCAndqNVgvVNa9/ZcGi29S7GuSgJeHqxtqgc75FmPxyrBooCINJbWufNH3DU l7vCE6ncN29BiPsgkgimLiGCyKHherVdm3BiIJMPuIEZR+OZedur6FvxkM35IbQH0cfWTqcnP+mGlIPsFGqoTOJFEnnbx7EyBw41AXxIMn03qoVwjlqxYAoC LamsgCcwqLqP7zQ2NMg1FA== X-Rspamd-Queue-Id: 4RQZH80H3Yz4rCK X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US] Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0 message dated "Tue, 15 Aug 2023 17:18:37 -0400." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii -------- In message , Ed Maste writes: > FreeBSD currently uses 9600 bps as the default for serial > communication -- in the boot loader, kernel serial console, /etc/ttys, > and so on. This was consistent with most equipment in the 90s, when > these defaults were established. Today 115200 bps seems to be much > more common, and I'm proposing that we make it the default for FreeBSD > 14.0. > > I have a review open: https://reviews.freebsd.org/D36295. There are a > few minor nits in the review to be addressed still but assuming > there's general agreement I'll iterate on those and commit this in a > few logical chunks. > There should probably be an UPDATING entry for those who use boot0 to revert back to 9600 in that case. From nobody Wed Aug 16 04:31:38 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQZwJ1MNjz4qB7q for ; Wed, 16 Aug 2023 04:31:44 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQZwG6SY2z4tbZ for ; Wed, 16 Aug 2023 04:31:42 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.48.228] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qW8C3-0007mV-P7 for freebsd-current@freebsd.org; Wed, 16 Aug 2023 06:31:39 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37G4VcED005319 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 16 Aug 2023 06:31:39 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37G4Vc8A005318 for freebsd-current@freebsd.org; Wed, 16 Aug 2023 06:31:38 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Wed, 16 Aug 2023 06:31:38 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: www/chromium will not build on a host w/ 8 CPU and 16G mem Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.48.228 X-Spamd-Result: default: False [-2.76 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.962]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RQZwG6SY2z4tbZ I have built ~2200 ports successful on my build server, which is a Dell R210 with 8x 3.30GHz CPU and 15.8 GB memory: Aug 11 19:03:21 jet kernel: CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz (3292.74-MHz K8-class CPU) Aug 11 19:03:21 jet kernel: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs Aug 11 19:03:21 jet kernel: avail memory = 16582250496 (15814 MB) I have set swap to 4GB + 10GB + 10GB: # swapctl -lh Device: Bytes Used: /dev/da0p3 4.0G 1.5G /dev/md9 10G 1.5G /dev/md10 10G 1.5G and poudriere does use ZFS. Building the last outstanding port www/chromium in single job mode fails reproducible with a kernel message: Aug 16 06:14:04 jet kernel: pid 48725 (ninja), jid 3, uid 65534, was killed: a thread waited too long to allocate a page What could I do in the port's options or kernel values? matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Wed Aug 16 06:19:37 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQdK52kjkz4qHYt for ; Wed, 16 Aug 2023 06:19:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQdK40h4Jz3KGM for ; Wed, 16 Aug 2023 06:19:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=oxVj45sb; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692166790; bh=8d/D5qddQBAkgwWf27a/JBW8msJU3+89TWCyJPMpy7U=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=oxVj45sb06vuj4eF2kB0yF/LUVznPPg7OZt2ETVLGV/FvpTdXS7Pl0P/60BGokWUziV4VE77yoDvW/dL40ef+M+qk8cXfKJpqYgv8jRDL+iL6s19S+ij8IPzKOKK8gteMDnrwhGUxiGV42R8g/N2OO3P8e2+YxFYM7nhcVD2EDicP5VPHHf1DSBrL5zir3LxYNmnO/wufLMeRda523v7m6qhenatbH+/lquUTUHfxmU7TllcnYZtXD8px2++ZJrkQfxoWYu1RbSra1kk3NrIWXpevRPz72HOfqAXPpm53U6zN4Z36/lDYDyHcgPm/ogNixOTH9HPurZcTHndJbzh4w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692166790; bh=+JNhxUmGvrVL5WIltrjQqzXQQekfFJCNJz/wAqkzDSi=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=KCqqekREZLZogaHi6ve3GE47S5vfXNt+cK2PHl3FzAqKzQEEtgvdC5122KgjqFmRLSf6fezCjYthWkRfqP1QHRW8jwI9iACxph3IAJmsGIHgNTSe6niXf2vI36AqKhnwtZtngxB+gGKADhJHrDa2ALDTdhmO1/KK6KYC7SJglFCzL3tMOeE0AAKlv41/O1DVNxWWCMSa5fzDWSP7nTaG2JpWLWJlPLuE0EWF2eKiWAPJBN7obGpMB1PKLdqgb7hx1tRF2LGGCIqrrZgyJZ3oXYoZ0E6J3KcZHwRNpKzoVdP/od6aEqGYUduH69mdOyYrZDrpBxdag1SQ4+9OULBYUQ== X-YMail-OSG: GVNDgtgVM1lujNnpkgOcQvB9BGbBHgE52MozTlTL5.cXVzOXCUp32UI7TsQsrqi pe0e.v8WolR9CY7Xzj..mkeXewrxzK24UOwKrnWCZW.cdmDPjCWxL0sDVmbCLX1o5uA4DxIDK0S4 Q.2w2myoo4Q0ekdtUu_ExFpOHbHSrHmZTKka1GGvGPk2EEU1PUH.wa67RGoV_ArHX2.JFEyVeCHo DYzqmd.QQUsZDA7G_P4GeWGBSaMib45h19dJQx.34TE5s9Mfi5lnAhdhbOwdENkQ72bk6JuoZjmi szG66r_cFiqisSJN6IzJ9LcoAkLE746vsxr4M.p_gDdKWgeZKu11nm.8OPThLyQ_LG4TlUWH.zBH bPz.f_0YmoUC9MatMgbNS3ljfqKbrC9243DzVRrPI_CzVEI5nWOwjtBnd7QApNhn5jp6InDQFegi gSUJpgO7mODxE5S7e5O36hrkPF3p_YxB03oEQeEC1urltwbdyDcqFUwaatgx.GcA2R0sgwe.MYT5 MTXPe3IqZWCMq9fVBf9rHHUXPNg4NTsszA85b12Ssj2JTo4xyBA7O2pep5WwvLlYgT6XtHgmrMMI 7BnQOKbkZyWkkPVrDj3iac5OqFm7BqQe7U7LwcgNm7WOr4jy7AUSywp9.PWeX2OM5d8aBszf7yXk UmiAuavFN0HABLQ6jqsHmJPGfaUJtipahAtaZfvJjtrtnYA4ytGp6JojyXF.orHNEZmoh6G.VcIF 1uWGpsCf9.cPK9na4XWMy5luP7pr787rbBGYWAtzsVG5rAVygJGhusa.tIpNBg8ICMtRlkqhgWG8 KeK63DcSMAmeHgwXMLiQgtAe9RX5Kma6DGbWGyKexyYV8OG5XNiRBXJRIC8P62YlYhW3lErS_GH2 P.wp5l6Oqghj.YognxKHRekfdONM4DUs4XKcYl8F0cUuNDCPkHb6yB5GcB47wYyZp4kCnpoAtxlH sh5diWfLUKSotiEPTcxQaBBnFQz6I9s3SzV47QD25PFnrApT6Z5qFhc5DxBdEmGlVtQ74i83WcxJ BmwkVLIYwRShws.uMGCRxUCBmF3XV8jMbvAm5nFrP6VXjOhWfuwUpy_O_LuoWao9w6VEa6UZfcxd FFDFZLpNc0MPHnyyJnvnfamGaFueQbsBaOc.jZKHiV32iZiUDLVB0TPtTH501DB8Cm00WHgS6JAC x8IaL2XChpakvndRKC8V45dB_S7.0f79974r5Kwb0LKmrwRxGFjC8lIJ9kXFxNF.4LCQGoxfwc0n TL29Qhj1S.4t1CwzbDZR5ZXajwfbNeFpfSdRDaDVUz2DhmN2L9zw5GfkMZ0P3u2WvtoD_42Dg3Gq UWEgM0m68tVWE_vCD3yhF6LbQLc_U7E6M1eUoTUY1MGdAxt6xQZAhiwuQRDWNWnoXsKUkuLp4sMr djPgnyQ09Bu3AbBwTHbiTORvhAWClIBm66pKX8v1vZUeqSrbOBsoh7ty.tYVF3qmyWg09cdsO7ek 8pVkeO80kJUEzoxs9nUtn9HIQantZ2Z5hFn8H2fBql9biGjngklc5psqsIMX_mZ6lhdqCfALF6nk XbyRl.DAnvLQcTCTpFcSLm07Vuur5uIrtPGGaMfYaAaW67w93vwIjbCuAk.NkKEG03UglIsWDxeK TzNNfeXFqCz.vnpIbH1Ya2d4a7YdBhulfpCky2NbpYMHNyD1PwdEZJ3rCY4qKKjvAV48ZBhLDwKI xgL9T9BwbGDz8kZH2C3k2U7YS.LWjF1NBkVfsc3ibIU_zwA1L2izleQs._xnykXV2.4.c7i1WnIL i..AON8Nwv8uD4wDh3bLn7zGxnFpniXIcbW6jx0NzxTkSav_kn9UXarLy3vYfd9KdXfuEy9YlDdX N0whj65O20P3wCFcKttlrsj9mpozVyKHXkfXMpLpxRtQ4bNpvnz6X8kT8wQrz1NqhMPVvYK1ZudS C0IlGYzpGf3JRo92ZsGezY5gRLga1IR8rZxqUnJ5XJOI9pqEFItAwO73c3teJ93NFtAlxAjEArBo LxFfckyP07NpVQkCYpjs4YI9gbHRMeF6NeIfMpK.mZTtoDczewm2Lf.eE69X1gTM4d9CSHX9ILni 2xa6yDhkbKkS_claC_c0xGCd0zF6gkYAKqchmHDDzhWaRRjp2zEartQOLzxB1sduKV2qMKAIbsJi WaaTQRPZOQ9rDFihlqc7d6dAppcTzMUDS_foGa8TEbDyjy.Iqz94H7bIcEsWRCUnKzexTvvK4eL3 CNjAxiwcpG7MKsDVCgGhVgpM6xApqvgLtx2Jdp8iZb4p1gDuJJsBb6GrwhxAGN0JZRnt7SxxLbrQ - X-Sonic-MF: X-Sonic-ID: 73a434c9-ed15-4eab-b165-cfeead51ae14 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Wed, 16 Aug 2023 06:19:50 +0000 Received: by hermes--production-gq1-6b7c87dcf5-m4lb7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID e9ac675689b03a331828f6f9d9aa649c; Wed, 16 Aug 2023 06:19:48 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: www/chromium will not build on a host w/ 8 CPU and 16G mem Message-Id: <2227F902-847E-4E50-B48A-B012CE51D96D@yahoo.com> Date: Tue, 15 Aug 2023 23:19:37 -0700 To: guru@unixarea.de, Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: <2227F902-847E-4E50-B48A-B012CE51D96D.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RQdK40h4Jz3KGM Matthias Apitz wrote on Date: Wed, 16 Aug 2023 04:31:38 UTC : > I have built ~2200 ports successful on my build server, which is a > Dell R210 with 8x 3.30GHz CPU and 15.8 GB memory: >=20 > Aug 11 19:03:21 jet kernel: CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ = 3.30GHz (3292.74-MHz K8-class CPU) > Aug 11 19:03:21 jet kernel: FreeBSD/SMP: Multiprocessor System = Detected: 8 CPUs > Aug 11 19:03:21 jet kernel: avail memory =3D 16582250496 (15814 MB) >=20 > I have set swap to 4GB + 10GB + 10GB: >=20 > # swapctl -lh > Device: Bytes Used: > /dev/da0p3 4.0G 1.5G > /dev/md9 10G 1.5G > /dev/md10 10G 1.5G Are those /dev/md* vnode backed? If not, what type are they? FYI: On 2017-Feb-13, at 7:20 PM, Konstantin Belousov wrote on the freebsd-arm list: QUOTE . . . swapfile write requires the write request to come through the filesystem write path, which might require the filesystem to allocate more memory and read some data. E.g. it is known that any ZFS write request allocates memory, and that write request on large UFS file might require allocating and reading an indirect block buffer to find the block number of the written block, if the indirect block was not yet read. As result, swapfile swapping is more prone to the trivial and = unavoidable deadlocks where the pagedaemon thread, which produces free memory, needs more free memory to make a progress. Swap write on the raw partition = over simple partitioning scheme directly over HBA are usually safe, while = e.g. zfs over geli over umass is the worst construction. END QUOTE Use of swap partitions is to be recommended to minimize the chance of paging related deadlocks. You have not reported on how much swap was in use shortly before the "was killed: a thread waited too long to allocate a page" notice. (After the notice is too late of a time frame to be of interest.) > and poudriere does use ZFS. >=20 > Building the last outstanding port www/chromium in single job mode > fails reproducible with a kernel message: >=20 > Aug 16 06:14:04 jet kernel: pid 48725 (ninja), jid 3, uid 65534, was = killed: > a thread waited too long to allocate a page You have not been explicit about how you have managed RAM+SWAP tradeoffs in /usr/local/etc/poudriere.conf settings. In /usr/local/etc/poudriere.conf : what are you using as the USE_TMPFS value: # Use tmpfs(5) # This can be a space-separated list of options: # wrkdir - Use tmpfs(5) for port building WRKDIRPREFIX # data - Use tmpfs(5) for poudriere cache/temp build data # localbase - Use tmpfs(5) for LOCALBASE (installing ports for = packaging/testing) # all - Run the entire build in memory, including builder jails. # yes - Enables tmpfs(5) for wrkdir and data # no - Disable use of tmpfs(5) Only 2 of the options keep the tmpfs use small: data no For example, building rust can use 10 GiBytes+ of just tmpfs space that competes for RAM+SWAP. The last personal I helped that was getting "a thread waited too long to allocate a page" it turned out that changing the USE_TMPFS=3D assignment to one of the 2 options eliminated the issue. (No guarnatee of such here.) [There were 2 USE_TMPFS=3D assignments, the 2nd "winning" --but the first had been intended.] Are you defining ALLOW_MAKE_JOBS=3D ? If yes, are you using /usr/local/etc/poudriere.d/make.conf (or the like) to assign MAKE_JOBS_NUMBER=3D (or the like)? If yes, to what? Last I knew the official port -> package builders use MAKE_JOBS_NUMBER=3D2 for their activity with ALLOW_MAKE_JOBS in use. A similar point goes for if you are assigning ALLOW_MAKE_JOBS_PACKAGES=3D . Are you? These can contribute to RAM+SWAP usage and higher load averages. > What could I do in the port's options or kernel values? I've not actually gone in either of those directions above. But nothing at this point says if I've happened to cover your case vs. not. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Aug 16 07:10:57 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQfSY70qSz4qLGf for ; Wed, 16 Aug 2023 07:11:25 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQfSY2xqSz3T68 for ; Wed, 16 Aug 2023 07:11:25 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37G7AwuV073005; Wed, 16 Aug 2023 16:10:58 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Wed, 16 Aug 2023 16:10:57 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Cc: Cy Schubert Subject: Re: Defaulting serial communication to 115200 bps for FreeBSD 14 Message-Id: <20230816161057.f8c6a3b4ebe9c672d920811e@dec.sakura.ne.jp> In-Reply-To: <20230816040257.A397366@slippy.cwsent.com> References: <20230816040257.A397366@slippy.cwsent.com> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4RQfSY2xqSz3T68 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] On Tue, 15 Aug 2023 21:02:57 -0700 Cy Schubert wrote: > Cheers, > Cy Schubert > FreeBSD UNIX: Web: https://FreeBSD.org > NTP: Web: https://nwtime.org > > e^(i*pi)+1=0 > > message dated "Tue, 15 Aug 2023 17:18:37 -0400." > Mime-Version: 1.0 > Content-Type: text/plain; charset=us-ascii > -------- > In message om> > , Ed Maste writes: > > FreeBSD currently uses 9600 bps as the default for serial > > communication -- in the boot loader, kernel serial console, /etc/ttys, > > and so on. This was consistent with most equipment in the 90s, when > > these defaults were established. Today 115200 bps seems to be much > > more common, and I'm proposing that we make it the default for FreeBSD > > 14.0. > > > > I have a review open: https://reviews.freebsd.org/D36295. There are a > > few minor nits in the review to be addressed still but assuming > > there's general agreement I'll iterate on those and commit this in a > > few logical chunks. > > > > There should probably be an UPDATING entry for those who use boot0 to > revert back to 9600 in that case. Not read the diff though, if the baud rate is re-initialized in boot1 or later (including btx, loader, kernel and userland) and handshake again, most of the boot process and later would run at 115200 bps. -- Tomoaki AOKI From nobody Wed Aug 16 07:30:46 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQfv103JCz4qMHk for ; Wed, 16 Aug 2023 07:30:53 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQfty48vnz3WSX for ; Wed, 16 Aug 2023 07:30:49 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp; dmarc=none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37G7UkEk075255 for ; Wed, 16 Aug 2023 16:30:47 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Wed, 16 Aug 2023 16:30:46 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: Re: www/chromium will not build on a host w/ 8 CPU and 16G mem Message-Id: <20230816163046.081be1181df386fae55065bc@dec.sakura.ne.jp> In-Reply-To: <2227F902-847E-4E50-B48A-B012CE51D96D@yahoo.com> References: <2227F902-847E-4E50-B48A-B012CE51D96D.ref@yahoo.com> <2227F902-847E-4E50-B48A-B012CE51D96D@yahoo.com> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [0.56 / 15.00]; AUTH_NA(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_SPAM_LONG(0.86)[0.863]; NEURAL_HAM_MEDIUM(-0.81)[-0.806]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[sakura.ne.jp]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Spamd-Bar: / X-Rspamd-Queue-Id: 4RQfty48vnz3WSX On Tue, 15 Aug 2023 23:19:37 -0700 Mark Millard wrote: > Matthias Apitz wrote on > Date: Wed, 16 Aug 2023 04:31:38 UTC : > > > I have built ~2200 ports successful on my build server, which is a > > Dell R210 with 8x 3.30GHz CPU and 15.8 GB memory: > > > > Aug 11 19:03:21 jet kernel: CPU: Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz (3292.74-MHz K8-class CPU) > > Aug 11 19:03:21 jet kernel: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs > > Aug 11 19:03:21 jet kernel: avail memory = 16582250496 (15814 MB) > > > > I have set swap to 4GB + 10GB + 10GB: > > > > # swapctl -lh > > Device: Bytes Used: > > /dev/da0p3 4.0G 1.5G > > /dev/md9 10G 1.5G > > /dev/md10 10G 1.5G > > Are those /dev/md* vnode backed? If not, what type are they? > > FYI: On 2017-Feb-13, at 7:20 PM, Konstantin Belousov > wrote on the freebsd-arm list: > > QUOTE > . . . > > swapfile write requires the write request to come through the filesystem > write path, which might require the filesystem to allocate more memory > and read some data. E.g. it is known that any ZFS write request > allocates memory, and that write request on large UFS file might require > allocating and reading an indirect block buffer to find the block number > of the written block, if the indirect block was not yet read. > > As result, swapfile swapping is more prone to the trivial and unavoidable > deadlocks where the pagedaemon thread, which produces free memory, needs > more free memory to make a progress. Swap write on the raw partition over > simple partitioning scheme directly over HBA are usually safe, while e.g. > zfs over geli over umass is the worst construction. > END QUOTE > > Use of swap partitions is to be recommended to minimize the chance of > paging related deadlocks. > > You have not reported on how much swap was in use shortly before the > "was killed: a thread waited too long to allocate a page" notice. > (After the notice is too late of a time frame to be of interest.) > > > and poudriere does use ZFS. > > > > Building the last outstanding port www/chromium in single job mode > > fails reproducible with a kernel message: > > > > Aug 16 06:14:04 jet kernel: pid 48725 (ninja), jid 3, uid 65534, was killed: > > a thread waited too long to allocate a page > > You have not been explicit about how you have managed > RAM+SWAP tradeoffs in /usr/local/etc/poudriere.conf > settings. > > In /usr/local/etc/poudriere.conf : what are you using > as the USE_TMPFS value: > > # Use tmpfs(5) > # This can be a space-separated list of options: > # wrkdir - Use tmpfs(5) for port building WRKDIRPREFIX > # data - Use tmpfs(5) for poudriere cache/temp build data > # localbase - Use tmpfs(5) for LOCALBASE (installing ports for packaging/testing) > # all - Run the entire build in memory, including builder jails. > # yes - Enables tmpfs(5) for wrkdir and data > # no - Disable use of tmpfs(5) > > Only 2 of the options keep the tmpfs use small: > > data > no > > For example, building rust can use 10 GiBytes+ of just tmpfs > space that competes for RAM+SWAP. > > The last personal I helped that was getting "a thread waited > too long to allocate a page" it turned out that changing the > USE_TMPFS= assignment to one of the 2 options eliminated the > issue. (No guarnatee of such here.) [There were 2 USE_TMPFS= > assignments, the 2nd "winning" --but the first had been > intended.] > > Are you defining ALLOW_MAKE_JOBS= ? If yes, are you using > /usr/local/etc/poudriere.d/make.conf (or the like) to assign > MAKE_JOBS_NUMBER= (or the like)? If yes, to what? Last I knew > the official port -> package builders use MAKE_JOBS_NUMBER=2 > for their activity with ALLOW_MAKE_JOBS in use. > > A similar point goes for if you are assigning > ALLOW_MAKE_JOBS_PACKAGES= . Are you? > > These can contribute to RAM+SWAP usage and higher load > averages. > > > What could I do in the port's options or kernel values? > > I've not actually gone in either of those directions above. > But nothing at this point says if I've happened to cover > your case vs. not. > > === > Mark Millard > marklmi at yahoo.com Just a FYI: www/chromium built fine (stable/13, though) with poudriere. RAM is 32GB and 64GB single dedicated swap partition on SSD. Using ports-mgmt/poudriere-devel. No ccache used. In /usr/local/etc/poudriere.conf, USE_TMPFS=yes ALLOW_MAKE_JOBS=yes The line below is kept commented out. #TMPFS_LIMIT=8 On main (booted from different physical drive on the same PC), I don't use poudriere[-devel], but www/chromium (previous version) built fine using ports-mgmt/pkg_replace. The size of /tmp (swap-backed tmpfs) is not limited (means at maximum of 64GB). -- Tomoaki AOKI From nobody Wed Aug 16 13:34:03 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQpy95T15z4qgZR for ; Wed, 16 Aug 2023 13:34:09 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQpy84CxZz4YPW for ; Wed, 16 Aug 2023 13:34:08 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.48.228] (helo=pureos) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qWGez-0004P2-NL for freebsd-current@freebsd.org; Wed, 16 Aug 2023 15:34:05 +0200 Date: Wed, 16 Aug 2023 15:34:03 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: Re: www/chromium will not build on a host w/ 8 CPU and 16G mem Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org References: <2227F902-847E-4E50-B48A-B012CE51D96D.ref@yahoo.com> <2227F902-847E-4E50-B48A-B012CE51D96D@yahoo.com> <20230816163046.081be1181df386fae55065bc@dec.sakura.ne.jp> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20230816163046.081be1181df386fae55065bc@dec.sakura.ne.jp> X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.48.228 X-Spamd-Result: default: False [1.76 / 15.00]; NEURAL_SPAM_MEDIUM(1.00)[0.996]; NEURAL_SPAM_SHORT(0.51)[0.506]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; NEURAL_HAM_LONG(-0.04)[-0.041]; REPLYTO_EQ_FROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[unixarea.de]; HAS_XOIP(0.00)[]; ARC_NA(0.00)[]; TO_DN_NONE(0.00)[]; BLOCKLISTDE_FAIL(0.00)[188.174.48.228:server fail,178.254.4.101:server fail]; FROM_HAS_DN(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: + X-Rspamd-Queue-Id: 4RQpy84CxZz4YPW Thanks for all the hints I got so far. I started already the build with MAKE_JOBS_UNSAFE=yes. It's running now for some 8 hours and has built 32% (as the log says). So it will need some 16 hours more... -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Wed Aug 16 16:48:00 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQvG8288xz4qr0f for ; Wed, 16 Aug 2023 16:48:16 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (bastille.leidinger.net [89.238.82.207]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQvG76tWNz3K5w for ; Wed, 16 Aug 2023 16:48:15 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; none Received: from webmail2.leidinger.net (roundcube.Leidinger.net [192.168.1.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: Alexander@Leidinger.net) by outgoing.leidinger.net (Postfix) with ESMTPSA id 2902159B; Wed, 16 Aug 2023 18:48:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1692204483; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4w0yYgiwwb1dOPSHF9IKJHnD2O5IEFufwWdP4EdgAFs=; b=iKXH6Oj2wSsWb0rxe0auCVJWUbCIy/yPC5oAM/FutcAkEavaj/SOOp6VivwKDBcVBX6DCb xxXnjdfoTrQm8cMAH+usx54TQDCGZAONNrE9TL7D7xY/X+eg+AzIYsto2Rs9/85ovuye6U mczxUmMZVTW1sDg1lVv9koBinuD3azqVZ3LbfuwJY0SloE+WxGf2EIFvJ5mjCeME2ZmQG0 o/c+AWQ3rK5qjqSxzpiPRc8AjeQVAZ0/7iCg+64KikNRboW1ph7q6F7fPLvVYwr4/nU+dZ fik6mC7AO96tBKcEDTQo3cy1r+iI3mX0WaeX2397yZ2wo3UQCo+vrfrsQmGnhg== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Wed, 16 Aug 2023 18:48:00 +0200 From: Alexander Leidinger To: Mateusz Guzik Cc: current@freebsd.org Subject: Re: Speed improvements in ZFS In-Reply-To: References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> Message-ID: <88e837aeb5a65c1f001de2077fb7bcbd@Leidinger.net> X-Sender: Alexander@Leidinger.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4RQvG76tWNz3K5w X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:34240, ipnet:89.238.64.0/18, country:DE] Am 2023-08-15 23:29, schrieb Mateusz Guzik: > On 8/15/23, Alexander Leidinger wrote: >> Am 2023-08-15 14:41, schrieb Mateusz Guzik: >> >>> With this in mind can you provide: sysctl kern.maxvnodes >>> vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes >>> vfs.recycles_free vfs.recycles >> >> After a reboot: >> kern.maxvnodes: 10485760 >> vfs.wantfreevnodes: 2621440 >> vfs.freevnodes: 24696 >> vfs.vnodes_created: 1658162 >> vfs.numvnodes: 173937 >> vfs.recycles_free: 0 >> vfs.recycles: 0 New values after one rund of periodic: kern.maxvnodes: 10485760 vfs.wantfreevnodes: 2621440 vfs.freevnodes: 356202 vfs.vnodes_created: 427696288 vfs.numvnodes: 532620 vfs.recycles_free: 20213257 vfs.recycles: 0 >>> Meanwhile if there is tons of recycles, you can damage control by >>> bumping kern.maxvnodes. What's the difference between recycles and recycles_free? Does the above count as bumping the maxvnodes? >> Looks like there are not much free directly after the reboot. I will >> check the values tomorrow after the periodic run again and maybe >> increase by 10 or 100 so see if it makes a difference. >> >>> If this is not the problem you can use dtrace to figure it out. >> >> dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or >> something else? >> > > I mean checking where find is spending time instead of speculating. > > There is no productized way to do it so to speak, but the following > crapper should be good enough: [script] I will let it run this night. Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From nobody Wed Aug 16 18:11:32 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RQx6K0h1tz4qwQ2 for ; Wed, 16 Aug 2023 18:11:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RQx6J3VM2z4G8l for ; Wed, 16 Aug 2023 18:11:36 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20221208.gappssmtp.com header.s=20221208 header.b=qjOoDGDT; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::52d) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-ed1-x52d.google.com with SMTP id 4fb4d7f45d1cf-5236b2b4cdbso8911870a12.3 for ; Wed, 16 Aug 2023 11:11:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1692209495; x=1692814295; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=xrdoMrvwMo4JCRJJv7XEuo/7Si3bJdZYD7YqhGRc1mE=; b=qjOoDGDTKWQtrCCl+qwrcpZ5iIvFCCCSR6Jqy1Y5nhI4qm/R7KJiG4NIKynxR5HGr6 MiVRRp+0nh0V3Hxy97pRRqrik7SRLCLPYXxAjLfFFioc6rMXZjfmD8FS9cwISc22Px4Y cMb9vMp6m+bxxomuaPpxqoHbwo9ZCHdKdYqXtdchGiuZfTvS4UAv2su263X6gzP+ga9A AzAwacTM+W5XIMcq6KFe5vsajf4UvodCx67F/uWcs6v3ZmrT/Ni+2AW0xALAy+2VRUwc 0WTKLQgnFChBLPcn7wGuGgTQrxOAU09T/Ud0Wr1qfAhau/WUkL4WDc3XiECVKPhkvPP1 SJGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692209495; x=1692814295; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=xrdoMrvwMo4JCRJJv7XEuo/7Si3bJdZYD7YqhGRc1mE=; b=TPMZ+iFHmyoN+Tx/Xpi2aO4hrChH5t4pwispzqpI+MYY04f+gCniaIOKwU9eoTH6rq u/H5fxVWp9kvmyP/dCYL/l6HRmoHOkV7WZPYzs/FBOortFDEtB81cG/rtFDXs/xjF+Uv EfV2iF8OQKxpSTPPFcHxGkVsm1I5xPI2+BjKgC/LDZzocfL3i6uqBoiUmudBjB0AbpHm SXop2M6YfGDPSv7TWIDFwKNbzrBXOEn8OejiICs2QaSCt6sOAJhjF9IsDD6yZCc/RwYC uG8SvuC3LUhRsx6Mk1Z+xhO4+6/mAZfIkGVyna/hPd/YYjN2ohOLXNb8PWpXVJwGN0gx wvOA== X-Gm-Message-State: AOJu0YwyhCbUp6NNKs/AJpmkbAvjH+mnkH6zFeML/MeVI1kFWfRjcBIE c0aLawpCFNv0KDVPjtA68GzkMTFRYqpB9RCLq0QN7KYAGweJH7Uk X-Google-Smtp-Source: AGHT+IF/SOhT5ja5snJ8Jav3eYoi8bkBo7orvQw9dz1IUfHvcKPnD9fYCOlGglI72jC1ghX2aCKFzwyhjYo2uUfbqkQ= X-Received: by 2002:aa7:d482:0:b0:523:38eb:395d with SMTP id b2-20020aa7d482000000b0052338eb395dmr2147050edr.9.1692209494681; Wed, 16 Aug 2023 11:11:34 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 From: Warner Losh Date: Wed, 16 Aug 2023 12:11:32 -0600 Message-ID: Subject: HEADS UP: $FreeBSD$ Removed from main To: FreeBSD Current Content-Type: multipart/alternative; boundary="000000000000d1017b06030e390a" X-Spamd-Result: default: False [-2.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; SUBJECT_HAS_CURRENCY(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20221208.gappssmtp.com:s=20221208]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52d:from]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_DN_ALL(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20221208.gappssmtp.com:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; RCPT_COUNT_ONE(0.00)[1]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com] X-Spamd-Bar: - X-Rspamd-Queue-Id: 4RQx6J3VM2z4G8l --000000000000d1017b06030e390a Content-Type: text/plain; charset="UTF-8" Greetings, I've just pushed the results that remove 31,035 instances of $FreeBSD$ from the main branch. >From this point out, we are effectively done with $FreeBSD$, though there's 4 places where $FreeBSD$ still exists in the tree: (1) In the contrib area, we have 621 remaining. I didn't remove any from contrib code. (2) There's 140 $FreeBSD$ or similar tags part of 'static const char rcsid[] = "$FreeBSD$". There's too many different styles to get with automation, and they are also intertwingled with sccsid[] entries, other SCCS tags and copyright strings. These will be sorted out separately, since we need to talk about old sccs tags in the tree and sort that out too at the same time. (3) There's one $FreeBSD$ in a version tag for bootinfo for chrp bootinfo.txt. I don't know the effects of removing this entry, so I left it in place. Not sure it's worth fixing. (4) indent tests remove $FreeBSD$ so that it doesn't screw them up. This is likely harmless, but could be removed. I didn't want to mess with it, though. I've removed 99.5%+ of the 'live' instances in the tree. The ones from contrib should be removed upstream and I plan to do a MFC-like thing where I cherry-pick some commits, and run my script against stable/13 and include the main hash so our scripting things it's a real MFC (the diff hashes will differ, so git's native tooling will take the slow path, at least, and may get confused otherwise. I've built world on aarch64 and amd64. I've built kernels on all the architectures after this change... or well, last night's main. I eye-balled today's changes and they all look good, but there's no incremental building with this change, so I'm starting a new universe after I loop the change back into my tree. Also: expect long build times, git fetch times, etc after this. Thanks to the many people who gave me feedback on the details of this change and how to chunk it up. Hopefully the 40ish commits was the right balance between 'all at once' and 'every dir'. As always, if there's any problems after this change, please let me know. Warner --000000000000d1017b06030e390a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Greetings,

I've just pus= hed the results that remove 31,035 instances of $FreeBSD$ from the main bra= nch.

From this point out, we are effectively done = with $FreeBSD$, though there's 4 places where $FreeBSD$ still exists in= the tree:
(1) In the contrib area, we have 621 remaining. I didn= 't remove any from contrib code.
(2) There's 140 $FreeBSD= $ or similar tags part of 'static const char rcsid[] =3D "$FreeBSD= $". There's too many different styles to get with automation, and = they are also intertwingled with sccsid[] entries, other SCCS tags and copy= right strings. These will be sorted out separately, since
we need= to talk about old sccs tags in the tree and sort that out too at the same = time.
(3) There's one $FreeBSD$ in a version tag for boot= info for chrp bootinfo.txt. I don't know the effects of removing this e= ntry, so I left it in place. Not sure it's worth fixing.
= (4) indent tests remove $FreeBSD$ so that it doesn't screw them up. Thi= s is likely harmless, but could be removed. I didn't want to mess with = it, though.

I've removed 99.5%+ of the 'li= ve' instances in the tree. The ones from contrib should be removed upst= ream and

I plan to do a MFC-like thing where = I cherry-pick some commits, and run my script against stable/13 and include= the main hash so our scripting things it's a real MFC (the diff hashes= will differ, so git's native tooling will take the slow path, at least= , and may get confused otherwise.

I've built w= orld on aarch64 and amd64. I've built kernels on all the architectures = after this change... or well, last night's main. I eye-balled today'= ;s changes and they all look good, but there's no incremental building = with this change, so I'm starting a new universe after I loop the chang= e back into my tree. Also: expect long build times, git fetch times, etc af= ter this.

Thanks to the many people who gave me fe= edback on the details of this change and how to chunk it up. Hopefully the = 40ish commits was the right balance between 'all at once' and '= every dir'.

As always, if there's any prob= lems after this change, please let me know.

Warner=
--000000000000d1017b06030e390a-- From nobody Wed Aug 16 22:14:36 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RR2Vs6XgBz4Twtp for ; Wed, 16 Aug 2023 22:14:45 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RR2Vr3qjzz4K1g for ; Wed, 16 Aug 2023 22:14:44 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=blastwave.org header.s=default header.b=i2xxE6x7; spf=pass (mx1.freebsd.org: domain of dclarke@blastwave.org designates 108.160.241.186 as permitted sender) smtp.mailfrom=dclarke@blastwave.org; dmarc=pass (policy=quarantine) header.from=blastwave.org Received: from [172.16.35.9] (cpe8c6a8d4d360a-cm8c6a8d4d3608.cpe.net.cable.rogers.com [99.253.151.152]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.16.1) with ESMTPSA id 37GMEb3i044629 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT) for ; Wed, 16 Aug 2023 18:14:38 -0400 (EDT) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1692224079; bh=BiN6Bk1n3iveABvGZtW1w9tLmsaa4zv6llqu+45fzrs=; h=Date:Subject:To:References:From:In-Reply-To; b=i2xxE6x76PoFIsgoHPyNBrk9FgPsKclEWAMYImDZLxnGL7wzLwEyOf+NAZyUT6hyd vQJq3L9R09RZ2xp68CMbJpN7RMkHj2csXFHnHY9VxFGsVoSxIj5wmX0r1AFsEVnks4 YDI4SmI0s0jV6ZOvxPUWzSchbDRJTFUkySQWqPBb5qkqmA7m+CSvMYHAmo6j8gfoaZ w3CWnD+/C0s6qL2qxytFPTAfQq1gRtezVj2BxaEjIhyKlJiogp0umH8dVSgARgrHK6 gsVO3XIzvduei8b6mDJIqmm9IMrutN7IVJ9kaCaprGK1LDIt+v836BJqDGGVIcX2xY Mb7YQWTfEel2w== Message-ID: <22dd3552-480b-05da-76d7-dfbe2cb03110@blastwave.org> Date: Wed, 16 Aug 2023 18:14:36 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: Defaulting serial communication to 115200 bps for FreeBSD 14 Content-Language: en-CA To: freebsd-current@freebsd.org References: <20230816040257.A397366@slippy.cwsent.com> <20230816161057.f8c6a3b4ebe9c672d920811e@dec.sakura.ne.jp> From: Dennis Clarke Organization: GENUNIX In-Reply-To: <20230816161057.f8c6a3b4ebe9c672d920811e@dec.sakura.ne.jp> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 37GMEb3i044629 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Spamd-Result: default: False [-4.70 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DMARC_POLICY_ALLOW(-0.50)[blastwave.org,quarantine]; R_DKIM_ALLOW(-0.20)[blastwave.org:s=default]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_IN_DNSWL_MED(-0.20)[108.160.241.186:from]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[blastwave.org:+]; HAS_ORG_HEADER(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4RR2Vr3qjzz4K1g On 8/16/23 03:10, Tomoaki AOKI wrote: > On Tue, 15 Aug 2023 21:02:57 -0700 > Cy Schubert wrote: > >> Cheers, >> Cy Schubert >> FreeBSD UNIX: Web: https://FreeBSD.org >> NTP: Web: https://nwtime.org >> >> e^(i*pi)+1=0 >> >> message dated "Tue, 15 Aug 2023 17:18:37 -0400." >> Mime-Version: 1.0 >> Content-Type: text/plain; charset=us-ascii >> -------- >> In message > om> >> , Ed Maste writes: >>> FreeBSD currently uses 9600 bps as the default for serial >>> communication -- in the boot loader, kernel serial console, /etc/ttys, >>> and so on. This was consistent with most equipment in the 90s, when >>> these defaults were established. Today 115200 bps seems to be much >>> more common, and I'm proposing that we make it the default for FreeBSD >>> 14.0. >>> >>> I have a review open: https://reviews.freebsd.org/D36295. There are a >>> few minor nits in the review to be addressed still but assuming >>> there's general agreement I'll iterate on those and commit this in a >>> few logical chunks. >>> >> >> There should probably be an UPDATING entry for those who use boot0 to >> revert back to 9600 in that case. > > Not read the diff though, if the baud rate is re-initialized in boot1 > or later (including btx, loader, kernel and userland) and handshake > again, most of the boot process and later would run at 115200 bps. > The default serial communications config on most telecom equipment that I have seen ( in the last forty years ) defaults to 9600 8n1. If people want something faster from FreeBSD then do the trivial : set comconsole_speed="115200" set console="comconsole" Is that not trivial enough? Also, merely a funny observation, if one tries a baud rate lower than 9600 then FreeBSD will panic. Jut a funny thing I have seen over and over. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional From nobody Thu Aug 17 02:28:37 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RR87r53gGz4q30W for ; Thu, 17 Aug 2023 02:28:40 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RR87r2BQNz4fMc for ; Thu, 17 Aug 2023 02:28:40 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-58cbdf3eecaso13134127b3.0 for ; Wed, 16 Aug 2023 19:28:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692239319; x=1692844119; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=XxWXN5Krp9xrKkqBhY1lwFOuViWW8aj7U6fNd/5RX9M=; b=c493fi9kpmnNAURKaW1mn48jPaFJXLTagHsTU2zrRaVGB9NIFDaJQTZIGYwM+nws2g 4ar65X07bGjs4nllNoMU5Oe8KDYhQ/+PIieWG/KsB5HmURdFSbUkVYogsVTcnboTG1oy 71VsdgFVKZpB4iLFd7hNXDnrO0VmEzT//Xyvw48tGpMYE9vo95dlZshCxP5zlkznGvzi vchpwJM6UWDgtVKyH6biNORBY/CnNCj3ptHFR8F+aDovqymDe8daEVSkE3uTLzf5DH1z 3lRnncthuVepyrP2G7dXUuF+j5couJy/pecY74sQp4RtvrELOWuCpUAGr9ZGXIXaKdBU NsZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692239319; x=1692844119; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XxWXN5Krp9xrKkqBhY1lwFOuViWW8aj7U6fNd/5RX9M=; b=D6Bri0SzWjUGH+nHJh0wrKk25RCuFbRhlxNNQRGOy6BqjfA99PRz+T1i+B4FlnUy0o IxbVuewyjkaHq+baXDCP6Yw/A6m2eFbdn1yEfNUBlnaYTA5t4/tZObMh6YyygehiWFAQ ExLEOSOt8EhTNZnJGCYP0etIA45I3ju/5+cMUuyUPB5ryv2hsvXCaEu+366v/5TUkObY HQCW0KArnPV2ukHCvneht+E8YS4SGZwT9tjNEdKxm7CaC8P4FtGbWsYaeZ4hcHZfS/Ta MqNvXAU7PA+7i7akrt6cPCr30EYw6zKVg4IIne2xxDjaKqI6TKN0avL4Ti6gWVTxQFAq zsfA== X-Gm-Message-State: AOJu0YwtEYSyIfcffQMbl4F79Dw0wdIw4a0DzEGwnUWLG3Y2orOTTuMt BalEXdSuIDqU+tu+SDc/ikNXDnWsce0= X-Google-Smtp-Source: AGHT+IHqpLtl87L5ULpUpgn5T/Apf+8CSsOA8yOG95H8WxSiLEP39MLFXyBVgjakHxyUPELX0tC+Hg== X-Received: by 2002:a81:7c8b:0:b0:589:d560:adaa with SMTP id x133-20020a817c8b000000b00589d560adaamr3132043ywc.7.1692239318902; Wed, 16 Aug 2023 19:28:38 -0700 (PDT) Received: from [192.168.1.66] (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id s3-20020a0de903000000b00583d44b4b30sm4362249ywe.99.2023.08.16.19.28.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Aug 2023 19:28:38 -0700 (PDT) Message-ID: <896a3247-1a60-fe7a-674e-92ae511b41aa@FreeBSD.org> Date: Wed, 16 Aug 2023 22:28:37 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: Defaulting serial communication to 115200 bps for FreeBSD 14 Content-Language: en-US To: Dennis Clarke , freebsd-current@freebsd.org References: <20230816040257.A397366@slippy.cwsent.com> <20230816161057.f8c6a3b4ebe9c672d920811e@dec.sakura.ne.jp> <22dd3552-480b-05da-76d7-dfbe2cb03110@blastwave.org> From: Alexander Motin In-Reply-To: <22dd3552-480b-05da-76d7-dfbe2cb03110@blastwave.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4RR87r2BQNz4fMc X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 16.08.2023 18:14, Dennis Clarke wrote: > The default serial communications config on most telecom equipment that > I have seen ( in the last forty years ) defaults to 9600 8n1. If people > want something faster from FreeBSD then do the trivial : > >     set comconsole_speed="115200" >     set console="comconsole" > > Is that not trivial enough? Except it is not a telecom equipment 40 years ago. Even at 115200 that I routinely use on my development systems I feel serial console output affects verbose boot time and kernel console debugging output. I also have BIOS console redirection enabled on my systems, and I believe the default there is also 115200, and even that is pretty slow. I see no point to stay compatible if it is unusable. -- Alexander Motin From nobody Thu Aug 17 03:37:43 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RR9gd0867z4q6f8 for ; Thu, 17 Aug 2023 03:37:49 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RR9gc5CR6z3HwQ; Thu, 17 Aug 2023 03:37:48 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.9] (cpe8c6a8d4d360a-cm8c6a8d4d3608.cpe.net.cable.rogers.com [99.253.151.152]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.16.1) with ESMTPSA id 37H3bhnf049010 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Wed, 16 Aug 2023 23:37:44 -0400 (EDT) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1692243465; bh=uE0rn7V+CaKpLc+TersSm5s8mkxMhQ7/S3+bU9EBozk=; h=Date:Subject:To:References:From:In-Reply-To; b=c9h9SmzfmQ30QzHYu7DujgCMNXg4ghammkZkrV8Zob+/BGXFwjr75XPRTcsCdn9I/ vei0Pg1KCclGQvl1yXEEgaSxPGNnBPAiO88iildAtDL2bbskSPYfUUl7neKMt+OmtQ 945CA4wLerxQWegFy0hzQgxcCn4THwJNqSm/ZSQMMBdF6Ir0C/iDqmrA8r43DeydIJ gSJ4xdazntOi9PSCRx79Ci4CQAKZHQVYLDQLQIwXVwVvN3gys7KHWA0vHoCfd0x78S HPSeZxRzwsYU4KrQSVji0+j43jMaKndkIom92k7LAqrM1gawIQVZ8RGIyBGRmzAnWK 2QKMBNEfqx05g== Message-ID: Date: Wed, 16 Aug 2023 23:37:43 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: Defaulting serial communication to 115200 bps for FreeBSD 14 Content-Language: en-CA To: Alexander Motin , freebsd-current@freebsd.org References: <20230816040257.A397366@slippy.cwsent.com> <20230816161057.f8c6a3b4ebe9c672d920811e@dec.sakura.ne.jp> <22dd3552-480b-05da-76d7-dfbe2cb03110@blastwave.org> <896a3247-1a60-fe7a-674e-92ae511b41aa@FreeBSD.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: <896a3247-1a60-fe7a-674e-92ae511b41aa@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 37H3bhnf049010 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Queue-Id: 4RR9gc5CR6z3HwQ X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] On 8/16/23 22:28, Alexander Motin wrote: > On 16.08.2023 18:14, Dennis Clarke wrote: >> The default serial communications config on most telecom equipment that >> I have seen ( in the last forty years ) defaults to 9600 8n1. If people >> want something faster from FreeBSD then do the trivial : >> >>      set comconsole_speed="115200" >>      set console="comconsole" >> >> Is that not trivial enough? > > Except it is not a telecom equipment 40 years ago.  Even at 115200 that > I routinely use on my development systems I feel serial console output > affects verbose boot time and kernel console debugging output.  I also > have BIOS console redirection enabled on my systems, and I believe the > default there is also 115200, and even that is pretty slow.  I see no > point to stay compatible if it is unusable. > You seem to be missing the point. You need to make a configuration choice. You. Not the world. You. Edit your /boot/loader.conf and put in the lines above. Then be happy. -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional PS: a recent CISCO ASA fireware defaults to 9600 8n1. Same as a lot of equipment. From nobody Thu Aug 17 04:40:51 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRC4j0Jp9z4qBPT for ; Thu, 17 Aug 2023 04:41:09 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRC4h48mPz3QK2 for ; Thu, 17 Aug 2023 04:41:08 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x12f.google.com with SMTP id 2adb3069b0e04-4fe7e67cc77so11883098e87.2 for ; Wed, 16 Aug 2023 21:41:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1692247263; x=1692852063; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ajkmVda08kMSfR+3kYJXeyJyXunY1epZ8DmIz0bZe0s=; b=KrS6e5LI0Uw4U2tlq/T4aDD02BGjFD++0JxTJKofjP1DFGj5k308b2gXjmN7wF9eaf buSkxN7bDolfDNGLcZES+AVLyZHMt1hlSNsp7o3LMg5v1kTm0DrJs+fiiOXbmE1Woky4 pDmgknRDxv2I0SdE0xtPE5oyMqv+mbG9dkzn8Clih+oK4reZ0Zbtw/4RvkAhyOF1wO1m pltK2OlCZWMAnlCKYJle1i9CvcMG8fZfx/D+ozHmAPoYgXawctk2zEZsTIZWA5KDHX5f 5LPQzhBJbmkN2TemDgdTUsWvrgQtz17tSVKqjELNRdYCNvwCHxbgTQJ1br7Dxn6nE1mu HGHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692247263; x=1692852063; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ajkmVda08kMSfR+3kYJXeyJyXunY1epZ8DmIz0bZe0s=; b=Qr/qdoaA8UoExeIRLLHGffsnvCn5wUQPF1Y3S/pA5vvz31WRPPQEbp56vDcnETc9eB 3cNR231loVUDzLR2kB/l/RulN+69Pk8BVRIWqgzA7HbhDjsjmP8vMaEeZyDKDYAg2qkJ rpEu++LpxDY9uB0d/FDxp4uoFLr+q/JFPCXZVX8O+ON0IgbLJeEy0ab1I+8Hnw+J8oSr qsXpBr8HF5Pann2wTTi6cP+1vt1uJMcqjl3iOxRtlmQxqL393CqsZZ8mAAUziOU8ksuA XnsUTvKX6Q05sCGLdY22Dp+kiXfKckv24HkW+m62pvBDZob+oTI65HctsXd9lDIuN9rE WNSw== X-Gm-Message-State: AOJu0YxKDSMxsqG1E4O3YyAPRZ2ydB6ob707EuFTmNlPkJEq3vbynek8 SU/u/WVybfQm563aftmQw8l/YJGZy7BLGUe0PChyrg== X-Google-Smtp-Source: AGHT+IEa+ZMYeokeJyNs/ozgrFvsG5MVYBZY87EJA/nIGNkNhCAEHsZOGGn6u8uYK7f6gPXMr3vwjtG7bEyP5XQUVK0= X-Received: by 2002:a05:6512:b83:b0:4fb:caed:95c3 with SMTP id b3-20020a0565120b8300b004fbcaed95c3mr3702396lfv.53.1692247262715; Wed, 16 Aug 2023 21:41:02 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <20230816040257.A397366@slippy.cwsent.com> <20230816161057.f8c6a3b4ebe9c672d920811e@dec.sakura.ne.jp> <22dd3552-480b-05da-76d7-dfbe2cb03110@blastwave.org> <896a3247-1a60-fe7a-674e-92ae511b41aa@FreeBSD.org> In-Reply-To: From: Warner Losh Date: Wed, 16 Aug 2023 22:40:51 -0600 Message-ID: Subject: Re: Defaulting serial communication to 115200 bps for FreeBSD 14 To: Dennis Clarke Cc: Alexander Motin , FreeBSD Current Content-Type: multipart/alternative; boundary="000000000000f77f430603170430" X-Rspamd-Queue-Id: 4RRC4h48mPz3QK2 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] --000000000000f77f430603170430 Content-Type: text/plain; charset="UTF-8" On Wed, Aug 16, 2023, 9:38 PM Dennis Clarke wrote: > On 8/16/23 22:28, Alexander Motin wrote: > > On 16.08.2023 18:14, Dennis Clarke wrote: > >> The default serial communications config on most telecom equipment that > >> I have seen ( in the last forty years ) defaults to 9600 8n1. If people > >> want something faster from FreeBSD then do the trivial : > >> > >> set comconsole_speed="115200" > >> set console="comconsole" > >> > >> Is that not trivial enough? > > > > Except it is not a telecom equipment 40 years ago. Even at 115200 that > > I routinely use on my development systems I feel serial console output > > affects verbose boot time and kernel console debugging output. I also > > have BIOS console redirection enabled on my systems, and I believe the > > default there is also 115200, and even that is pretty slow. I see no > > point to stay compatible if it is unusable. > > > > You seem to be missing the point. > > You need to make a configuration choice. You. Not the world. You. > > Edit your /boot/loader.conf and put in the lines above. > > Then be happy. > > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken > GreyBeard and suspenders optional > > PS: a recent CISCO ASA fireware defaults to 9600 8n1. Same as a lot of > equipment. > Yes. Some tiny number of things has that as a default, an even larger number of things have a default of 115200 or even faster. And have had that default since the 90s. The whole point of defaults is that they reflect the needs of the most people. FreeBSD's defaults were already starting to be dated in 1.0... today almost everyone changes the defaults to the new value we are advocating. This is to make FreeBSD more useful out of the box to more people. To turn your argument around: people wanting the old defaults can configure their systems easily enough. If we look purely at the numbers, vastly fewer people withh be inconvenienced at 115200 than at 9600. People can still use 9600... that's likely never going away... this is just a more sensible default. Warner > --000000000000f77f430603170430 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Wed, Aug 16, 2023, 9:38 PM Dennis Clarke <dclarke@blastwave.org> wrote:
=
On 8/16/23 22:28, Alexander Motin wrot= e:
> On 16.08.2023 18:14, Dennis Clarke wrote:
>> The default serial communications config on most telecom equipment= that
>> I have seen ( in the last forty years ) defaults to 9600 8n1. If p= eople
>> want something faster from FreeBSD then do the trivial :
>>
>> =C2=A0=C2=A0=C2=A0=C2=A0 set comconsole_speed=3D"115200"=
>> =C2=A0=C2=A0=C2=A0=C2=A0 set console=3D"comconsole"
>>
>> Is that not trivial enough?
>
> Except it is not a telecom equipment 40 years ago.=C2=A0 Even at 11520= 0 that
> I routinely use on my development systems I feel serial console output=
> affects verbose boot time and kernel console debugging output.=C2=A0 I= also
> have BIOS console redirection enabled on my systems, and I believe the=
> default there is also 115200, and even that is pretty slow.=C2=A0 I se= e no
> point to stay compatible if it is unusable.
>

You seem to be missing the point.

You need to make a configuration choice. You. Not the world. You.

Edit your /boot/loader.conf and put in the lines above.

Then be happy.

--
Dennis Clarke
RISC-V/SPARC/PPC/ARM/CISC
UNIX and Linux spoken
GreyBeard and suspenders optional

PS: a recent CISCO ASA fireware defaults to 9600 8n1. Same as a lot of
=C2=A0 =C2=A0 =C2=A0equipment.

Yes. Some tiny number of things has that as a= default, an even larger number of things have a default of 115200 or even = faster. And have had that default since the 90s. The whole point of default= s is that they reflect the needs of the most people. FreeBSD's defaults= were already starting to be dated in 1.0...=C2=A0 today almost everyone ch= anges the defaults to the new value we are advocating. This is to make Free= BSD more useful out of the box to more people. To turn your argument around= : people wanting the old defaults can configure their systems easily enough= . If we look purely at the numbers, vastly fewer people withh be inconvenie= nced at 115200 than at 9600. People can still use 9600... that's likely= never going away... this is just a more sensible default.=C2=A0

Warner
--000000000000f77f430603170430-- From nobody Thu Aug 17 10:45:06 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRM8f3mSdz4qWcH for ; Thu, 17 Aug 2023 10:45:06 +0000 (UTC) (envelope-from ronald@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRM8f3Gzjz4T1q for ; Thu, 17 Aug 2023 10:45:06 +0000 (UTC) (envelope-from ronald@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692269106; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=eqACW/xoZnjWBRW6r2Ya1Sn4D+PY/PkjSQD/tsG1rsA=; b=knHn38SP5L7APWe29zu7r3ejy1FwNb1/vNKJTiP6IbcQiK4SDVeeBw+5Ch0AeGqm7YCVyw QiGznGJTJpoX2wEfr1YK4++OrhkkjvAGWbLvX+UjxaueajLTFk3H6LVH1IvigXokCzDXIM 2v+7/bN9JwKA1ao58M04JwbbHoyJWHsrbtsjoKnTfF0CUP0Tjp8tJtRw/h9N1nlv45QiSP MRKUPiC++SRDkHVJ4dSFB0bL64+lA5AYT35yEgdp6lA28sxDSRtbnkJJP6rwVm5RmUsCXZ vHmQpo4xceYtCZNBfs+yVyTC+ZLWv+T26HZv3oPXtrJGW8JmpUrHupwuMJsx4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692269106; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=eqACW/xoZnjWBRW6r2Ya1Sn4D+PY/PkjSQD/tsG1rsA=; b=qC/1NM7yvWVMgWSBykUFVVmj6i9z52TG7LbeemRvnMkDvd5RVjMzQj9uIkcsTlSFWjyGyp k3ZF0qAd0nIVYkQvZTvJMjxrzTtXPh8T8PB+Djc6qdiqTwaIIkXiSw6+qht/Jzd22GxVxh ppza4q0NXNzFrBrHMFu4TWpvm6CcugaNex1DvlqDyZYlY1HR8Vfkm1lYdrFBa+rZW6R8DO AUG2yU+s2PFPbv03xnwBOKLvXzrvDYQqkcmXUarD4yAqB9G+wbP891VLaligU44SvC3D/z MZC5MYcMCysJwwcHRJotADWEjWbJxwNssPVxwJbdx3MaT+AUfyid67QprSinZA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692269106; a=rsa-sha256; cv=none; b=n2IED63dNselCC0Efoqd+CvnCptM4C7RrKc2eek8KxO+wBoTEEyA6wY3YdcRzR7G42+nF7 Jw6vWzbZPvaUbesIbTCNgxYHUwpWtHLGvocrzdqMFxYd51UbM6hoJFNz7ux2eAgizEdHrC 2aBWW0OC16TiWKwV2cX5DoMapVLpQj83oKSdjgqTXNr1K38je1ame/9ET/fwrw5TNq3N+E dHJspz/MYQbPdWWvxtj00smY5nJ7A4Le5Ys06ZZri3ezuTSBbtr1vmZP7MnAMuqiznsvB6 Woh6umhLV+VdUZLud1V2WOoMoRC19XOIll9BxBdczk0GNSb3eVE+NyBryM2jkA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from [192.168.1.109] (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: ronald/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RRM8f0WmQzjw9 for ; Thu, 17 Aug 2023 10:45:05 +0000 (UTC) (envelope-from ronald@FreeBSD.org) Message-ID: <76ecade1-3f42-a314-825d-6f1ba0b2dcaa@FreeBSD.org> Date: Thu, 17 Aug 2023 12:45:06 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Content-Language: en-US To: freebsd-current@freebsd.org From: Ronald Klop Subject: building with llvm16 pkg fails in tests Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 SGksDQoNClRvIHNhdmUgdGltZSBvbiBteSBSYXNwYmVycnkgUGkgSSB3b3VsZCBsaWtlIHRv IGJ1aWxkIEZyZWVCU0QgdXNpbmcgYSBsbHZtIHBrZyBpbnN0ZWFkIG9mIGxsdm0gaW4gdGhl IHRyZWUuDQoNCk15IC9ldGMvbWFrZS5jb25mOg0KV0lUSE9VVF9UT09MQ0hBSU49eWVzDQpM RD0vdXNyL2xvY2FsL2xsdm0xNi9iaW4vbGQubGxkDQpDQz0vdXNyL2xvY2FsL2xsdm0xNi9i aW4vY2xhbmcNCkNYWD0vdXNyL2xvY2FsL2xsdm0xNi9iaW4vY2xhbmcrKw0KQ1BQPS91c3Iv bG9jYWwvbGx2bTE2L2Jpbi9jbGFuZy1jcHANCk9CSkNPUFk9L3Vzci9sb2NhbC9sbHZtMTYv YmluL2xsdm0tb2JqY29weQ0KDQojV0lUSE9VVF9DTEVBTj15ZXMNCg0KDQpUaGlzIGZhaWxz IGluOg0KDQovdXNyL2xvY2FsL2xsdm0xNi9iaW4vY2xhbmcrKyAtTzIgLXBpcGUgLWZuby1j b21tb24gLWZQSUUgLVduby1mb3JtYXQtemVyby1sZW5ndGggLW5vYnVpbHRpbmluYyAtaWRp cmFmdGVyIC91c3IvbG9jYWwvbGx2bTE2L2xpYi9jbGFuZy8xNi9pbmNsdWRlIC1mc3RhY2st cHJvdGVjdG9yLXN0cm9uZyAtV3N5c3RlbS1oZWFkZXJzIC1XZXJyb3IgLVdhbGwgLVduby1m b3JtYXQteTJrIC1XIC1Xbm8tdW51c2VkLXBhcmFtZXRlciAtV3BvaW50ZXItYXJpdGggLVdy ZXR1cm4tdHlwZSAtV2Nhc3QtcXVhbCAtV3dyaXRlLXN0cmluZ3MgLVdzd2l0Y2ggLVdzaGFk b3cgLVd1bnVzZWQtcGFyYW1ldGVyIC1XY2FzdC1hbGlnbiAtV2NoYXItc3Vic2NyaXB0cyAt V2RhdGUtdGltZSAtV21pc3NpbmctdmFyaWFibGUtZGVjbGFyYXRpb25zIC1Xbm8tZW1wdHkt Ym9keSAtV25vLXN0cmluZy1wbHVzLWludCAtV25vLXVudXNlZC1jb25zdC12YXJpYWJsZSAt V25vLWVycm9yPXVudXNlZC1idXQtc2V0LXBhcmFtZXRlciAtTzAgLWcwIC1RdW51c2VkLWFy Z3VtZW50cyAtSS91c3Ivb2JqL3Vzci9zcmMvYW1kNjQuYW1kNjQvdG1wL3Vzci9pbmNsdWRl L3ByaXZhdGUgLUkvdXNyL3NyYy9jb250cmliL2dvb2dsZXRlc3QvZ29vZ2xlbW9jay9pbmNs dWRlIC1JL3Vzci9zcmMvY29udHJpYi9nb29nbGV0ZXN0L2dvb2dsZW1vY2sgLUkvdXNyL3Ny Yy9jb250cmliL2dvb2dsZXRlc3QvZ29vZ2xldGVzdC9pbmNsdWRlIC1JL3Vzci9zcmMvY29u dHJpYi9nb29nbGV0ZXN0L2dvb2dsZXRlc3QgLUkvdXNyL29iai91c3Ivc3JjL2FtZDY0LmFt ZDY0L3RtcC91c3IvaW5jbHVkZS9wcml2YXRlIC1ER1RFU1RfSEFTX1BPU0lYX1JFPTEgLURH VEVTVF9IQVNfUFRIUkVBRD0xIC1ER1RFU1RfSEFTX1NUUkVBTV9SRURJUkVDVElPTj0xIC1m cnR0aSAtV25vLWRlcHJlY2F0ZWQtY29weSAtV25vLXNpZ25lZC11bnNpZ25lZC13Y2hhciAt REdURVNUX0hBU19QT1NJWF9SRT0xIC1ER1RFU1RfSEFTX1BUSFJFQUQ9MSAtREdURVNUX0hB U19TVFJFQU1fUkVESVJFQ1RJT049MSAtZnJ0dGkgLVduby1kZXByZWNhdGVkLWNvcHkgLVdu by1zaWduZWQtdW5zaWduZWQtd2NoYXIgLWZQSUUgLXN0ZD1jKysxNCAtV25vLWRlcHJlY2F0 ZWQtY29weSAtV25vLWVycm9yPWluY29uc2lzdGVudC1taXNzaW5nLW92ZXJyaWRlIC1Xbm8t ZXJyb3I9bWlzc2luZy12YXJpYWJsZS1kZWNsYXJhdGlvbnMgLVduby1lcnJvcj1zaWduLWNv bXBhcmUgLVduby1lcnJvcj11bnVzZWQtcGFyYW1ldGVyIC1Xbm8tYysrMTEtZXh0ZW5zaW9u cyAgLVdsLC16cmVscm8gLXBpZSAgLS1sZC1wYXRoPS91c3IvbG9jYWwvbGx2bTE2L2Jpbi9s ZC5sbGQgLW8gZ21vY2stYWN0aW9uc190ZXN0ICBnbW9jay1hY3Rpb25zX3Rlc3QubyAtbHBy aXZhdGVnbW9ja19tYWluIC1scHJpdmF0ZWdtb2NrIC1scHJpdmF0ZWd0ZXN0DQpsZC5sbGQ6 IGVycm9yOiB1bmRlZmluZWQgc3ltYm9sOiB0ZXN0aW5nOjppbnRlcm5hbDo6RGVhdGhUZXN0 OjpDcmVhdGUoY2hhciBjb25zdCosIHRlc3Rpbmc6Ok1hdGNoZXI8c3RkOjpfXzE6OmJhc2lj X3N0cmluZzxjaGFyLCBzdGQ6Ol9fMTo6Y2hhcl90cmFpdHM8Y2hhcj4sIHN0ZDo6X18xOjph bGxvY2F0b3I8Y2hhcj4+IGNvbnN0Jj4sIGNoYXIgY29uc3QqLCBpbnQsIHRlc3Rpbmc6Omlu dGVybmFsOjpEZWF0aFRlc3QqKikNCj4+PiByZWZlcmVuY2VkIGJ5IGdtb2NrLWFjdGlvbnNf dGVzdC5jYw0KPj4+ICAgICAgICAgICAgICAgZ21vY2stYWN0aW9uc190ZXN0Lm86KHRlc3Rp bmc6Oihhbm9ueW1vdXMgbmFtZXNwYWNlKTo6QnVpbHRJbkRlZmF1bHRWYWx1ZURlYXRoVGVz dF9Jc1VuZGVmaW5lZEZvclJlZmVyZW5jZXNfVGVzdDo6VGVzdEJvZHkoKSkNCj4+PiByZWZl cmVuY2VkIGJ5IGdtb2NrLWFjdGlvbnNfdGVzdC5jYw0KPj4+ICAgICAgICAgICAgICAgZ21v Y2stYWN0aW9uc190ZXN0Lm86KHRlc3Rpbmc6Oihhbm9ueW1vdXMgbmFtZXNwYWNlKTo6QnVp bHRJbkRlZmF1bHRWYWx1ZURlYXRoVGVzdF9Jc1VuZGVmaW5lZEZvclJlZmVyZW5jZXNfVGVz dDo6VGVzdEJvZHkoKSkNCj4+PiByZWZlcmVuY2VkIGJ5IGdtb2NrLWFjdGlvbnNfdGVzdC5j Yw0KPj4+ICAgICAgICAgICAgICAgZ21vY2stYWN0aW9uc190ZXN0Lm86KHRlc3Rpbmc6Oihh bm9ueW1vdXMgbmFtZXNwYWNlKTo6QnVpbHRJbkRlZmF1bHRWYWx1ZURlYXRoVGVzdF9Jc1Vu ZGVmaW5lZEZvck5vbkRlZmF1bHRDb25zdHJ1Y3RpYmxlVHlwZV9UZXN0OjpUZXN0Qm9keSgp KQ0KPj4+IHJlZmVyZW5jZWQgNCBtb3JlIHRpbWVzDQoNCmxkLmxsZDogZXJyb3I6IHVuZGVm aW5lZCBzeW1ib2w6IHRlc3Rpbmc6OkV4cGVjdGF0aW9uOjpFeHBlY3RhdGlvbihzdGQ6Ol9f MTo6c2hhcmVkX3B0cjx0ZXN0aW5nOjppbnRlcm5hbDo6RXhwZWN0YXRpb25CYXNlPiBjb25z dCYpDQoNCg0KQW55IHRob3VnaHRzIG9uIGhvdyB0byBmaXggdGhpcz8NCkNvbXBpbGluZyB3 aXRoIHRoZSBpbiB0cmVlIGxsdm0gZG9lcyB3b3JrIHByb3Blcmx5Lg0KDQpOQjogYnVpbGRp bmcgMTQtQ1VSUkVOVCBpbiBhIDE0LUNVUlJFTlQgamFpbCBvbiAxMy4yLVJFTEVBU0UuIEFu ZCB0aGlzIGV4YW1wbGUgaXMgbXkgdGVzdCBvbiBhbWQ2NCB0byBtYWtlIGl0IHdvcmsgYmVm b3JlIEkgd2lsbCBkbyB0aGlzIGluIHRoZSBzbG93IFJQSTQuDQoNClJlZ2FyZHMsDQpSb25h bGQuDQo= From nobody Thu Aug 17 12:47:55 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRPtY5xQdz4qdKk for ; Thu, 17 Aug 2023 12:48:05 +0000 (UTC) (envelope-from dclarke@blastwave.org) Received: from mail.oetec.com (mail.oetec.com [108.160.241.186]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mail.oetec.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRPtY1W0mz4dRc; Thu, 17 Aug 2023 12:48:05 +0000 (UTC) (envelope-from dclarke@blastwave.org) Authentication-Results: mx1.freebsd.org; none Received: from [172.16.35.9] (cpe8c6a8d4d360a-cm8c6a8d4d3608.cpe.net.cable.rogers.com [99.253.151.152]) (authenticated bits=0) by mail.oetec.com (8.17.1/8.16.1) with ESMTPSA id 37HCluSg057175 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NOT); Thu, 17 Aug 2023 08:47:57 -0400 (EDT) (envelope-from dclarke@blastwave.org) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=blastwave.org; s=default; t=1692276478; bh=nP2OwXVEOACuFWftia5da610pwHuN2WJxSFUsHJ8PPk=; h=Date:Subject:To:Cc:References:From:In-Reply-To; b=qm9IROob9j13D3XlJNdp6vdvEnZZfpi+L/Yl0KX1z6zp0JpXftskaEAg8wJVdcrpU GHozV7iXDKs2rBd6rRKWsPSgDhicDrmL6hBTLULyPgncO4Dndi9qLpPxR//Yn/wfiz YQvCw/M0u1se2qXVOtBn/4fHxJnUaN2i1oPgm3VBbxXOY687DVTgLrJnjA1IB0ub/r PqTv5w0/NjazhL+UGr1XAH0LKcaAgoeanV7kvSVOZmEW3Wl9+Eaai9wzYfySsaXCvh TPrQzxvRUicwtmL/hjtBamXmpC0lU7u9LoC6aj+ypH9NZvM1cSWnrptvQuNcObS2nR WwnVsnVGBA3Bg== Message-ID: <0fd7ced1-93d5-a22d-6ddb-b2562eb7f17b@blastwave.org> Date: Thu, 17 Aug 2023 08:47:55 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: Defaulting serial communication to 115200 bps for FreeBSD 14 Content-Language: en-CA To: Warner Losh Cc: Alexander Motin , FreeBSD Current References: <20230816040257.A397366@slippy.cwsent.com> <20230816161057.f8c6a3b4ebe9c672d920811e@dec.sakura.ne.jp> <22dd3552-480b-05da-76d7-dfbe2cb03110@blastwave.org> <896a3247-1a60-fe7a-674e-92ae511b41aa@FreeBSD.org> From: Dennis Clarke Organization: GENUNIX In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-oetec-MailScanner-Information: Please contact the ISP for more information X-oetec-MailScanner-ID: 37HCluSg057175 X-oetec-MailScanner: Found to be clean X-oetec-MailScanner-From: dclarke@blastwave.org X-Spam-Status: No X-Rspamd-Queue-Id: 4RRPtY1W0mz4dRc X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:812, ipnet:108.160.240.0/20, country:CA] On 8/17/23 00:40, Warner Losh wrote: > On Wed, Aug 16, 2023, 9:38 PM Dennis Clarke wrote: > >> On 8/16/23 22:28, Alexander Motin wrote: >>> On 16.08.2023 18:14, Dennis Clarke wrote: >>>> The default serial communications config on most telecom equipment that >>>> I have seen ( in the last forty years ) defaults to 9600 8n1. If people >>>> want something faster from FreeBSD then do the trivial : >>>> >>>> set comconsole_speed="115200" >>>> set console="comconsole" >>>> >>>> Is that not trivial enough? >>> >>> Except it is not a telecom equipment 40 years ago. Even at 115200 that >>> I routinely use on my development systems I feel serial console output >>> affects verbose boot time and kernel console debugging output. I also >>> have BIOS console redirection enabled on my systems, and I believe the >>> default there is also 115200, and even that is pretty slow. I see no >>> point to stay compatible if it is unusable. >>> >> >> You seem to be missing the point. >> >> You need to make a configuration choice. You. Not the world. You. >> >> Edit your /boot/loader.conf and put in the lines above. >> >> Then be happy. >> >> -- >> Dennis Clarke >> RISC-V/SPARC/PPC/ARM/CISC >> UNIX and Linux spoken >> GreyBeard and suspenders optional >> >> PS: a recent CISCO ASA fireware defaults to 9600 8n1. Same as a lot of >> equipment. >> > > Yes. Some tiny number of things has that as a default, an even larger > number of things have a default of 115200 or even faster. And have had that > default since the 90s. The whole point of defaults is that they reflect the > needs of the most people. FreeBSD's defaults were already starting to be > dated in 1.0... today almost everyone changes the defaults to the new > value we are advocating. This is to make FreeBSD more useful out of the box > to more people. To turn your argument around: people wanting the old > defaults can configure their systems easily enough. If we look purely at > the numbers, vastly fewer people withh be inconvenienced at 115200 than at > 9600. People can still use 9600... that's likely never going away... this > is just a more sensible default. > > Warner > That makes perfect flawless sense to me. The logic of "popular" or "most valuable" to the most users for something like this. Yes I know that is a dangling elliptical sentence but it works. -- -- Dennis Clarke RISC-V/SPARC/PPC/ARM/CISC UNIX and Linux spoken GreyBeard and suspenders optional From nobody Thu Aug 17 18:07:06 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRXyn1S3gz4mTqZ for ; Thu, 17 Aug 2023 18:07:13 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-lf1-x12e.google.com (mail-lf1-x12e.google.com [IPv6:2a00:1450:4864:20::12e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRXyl5vFXz3MHd for ; Thu, 17 Aug 2023 18:07:11 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=otIHHJpa; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2a00:1450:4864:20::12e as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-x12e.google.com with SMTP id 2adb3069b0e04-4fe0d5f719dso13282966e87.2 for ; Thu, 17 Aug 2023 11:07:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692295630; x=1692900430; h=content-transfer-encoding:in-reply-to:subject:from:content-language :references:to:user-agent:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=mfO2W/ATjembkikgreiVXknkj+Z8RX9UKIA8MSflMiE=; b=otIHHJpacDA/jRRH1bMMJtdGH03jHAs+Lad3QUac5R3NuJXIGfqpD02SeWZtPLeYjr 7txI7w2cB8LOEbsbtds+MxJ/gWnrLmyOJWNIueQ22cTVfOfzu/uU7dvdt8oZtlRutjVW 4TiYvjsh/3MzBgL6dJrDZKuTIkmb3wyjHFZpq83RTzkN0f5zm5yfZnemYbOj/JTbnSSm JDdIv+6cHW+JI8VOTGa9ss2t5bGBi3qaCPU62P+VrcjoOuWN1+TQ4S2+6yXmnZSWvEM5 PPH/5Wf+hKi4vcvsYxn4HgTbrxYlUIJZBfTrt8/tSak2PyWdb5DhDcjiyPik1tSW76Vf dZ6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692295630; x=1692900430; h=content-transfer-encoding:in-reply-to:subject:from:content-language :references:to:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mfO2W/ATjembkikgreiVXknkj+Z8RX9UKIA8MSflMiE=; b=LLlDDEDJXfNaWYXfKuBbEwit/FTkR2JPDMbJU86olWLpEjRx3nmlQGfEpLZgR+U7vR 6qajjkXLRA083Jh5nN5MSGQ29E4ffzlYD2gJYf8rX6sdI5GJrsSGPeDSg67i030W/fCY xxHMsdXKaS87A5wabo/w1iEIMoiSTDz3eVhk5cU/AGWLfYjp3SktqS3f1FjgxCu6eGkC /p7u9hXF2Y6GtLuRmrRe9R3+zl5CcnHTWgXf7VQCVXCgcomz3X6wGT0CP7OnDA8vmes6 cS8P8495tJgJjh/MfJLoLzZIdnhG8V7SlJgtnDkYXoIVJXtnpLURGqJMViK6EL1U3+w9 a/Cg== X-Gm-Message-State: AOJu0YwEAxvuHO14jh96Agm921YbuQNGjxwyJR2bN2eTocTxple/dKj/ CfTaUNLO/i37CaUBtahDG0yibcvn4qk= X-Google-Smtp-Source: AGHT+IFXFB45g9X0TSBUlX9LjdO5gZZ3mPBfMcEUTF7JbBBR5X3IcGt3eEVfjimZvy2y4m/nWfxaWg== X-Received: by 2002:a05:6512:15a6:b0:4fe:8ba9:4c0 with SMTP id bp38-20020a05651215a600b004fe8ba904c0mr62943lfb.26.1692295630007; Thu, 17 Aug 2023 11:07:10 -0700 (PDT) Received: from [192.168.1.10] (80-42-66-93.dynamic.dsl.as9105.com. [80.42.66.93]) by smtp.gmail.com with ESMTPSA id k25-20020aa7c059000000b0052889d090bfsm46183edo.79.2023.08.17.11.07.08 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Aug 2023 11:07:09 -0700 (PDT) Message-ID: Date: Thu, 17 Aug 2023 19:07:06 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 To: Warner Losh , freebsd-current@freebsd.org References: <62d300c8-2c3e-58fa-334e-23a17962279a@freebsd.org> <753f3990-9903-3718-445c-49fc01f960a7@freebsd.org> Content-Language: en-US From: Graham Perrin Subject: Re: Fwd: Unreliability with DHCP In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.45 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.45)[-0.452]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[grahamperrin]; RCVD_VIA_SMTP_AUTH(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::12e:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RRXyl5vFXz3MHd On 06/08/2023 17:05, Warner Losh wrote: > On Sun, Aug 6, 2023 at 12:38 AM Kevin Oberman wrote: > > On Sat, Aug 5, 2023 at 3:16 PM Graham Perrin > wrote: > > On 05/08/2023 12:39, Oleksandr Kryvulia wrote: > > > … > > > > As dirty workaround I have in my /etc/rc.resume > > > > service netif restart > > service routing restart > > Thanks, I'll try when I'm next on campus. > > I do know that 'service routing restart' can be problematic. > Please, > see, for example, ; I had > something > similar a few minutes ago. > > > My usual solution is "service netif restart wlan0" (or the > interface you are using). It should restart the interface, if > rc.conf calls for it, dhcpclient and wpa_supplicant (if appropriate). > > > I'll have to remember that. I've been removing and reinsterting my usb > dongle when when dhclient fails. > I'd like to move to the internal wlan card, but the driver support has > some show-stopper issues with > suspend/resume for me more basic than dhclient... However, once those > are resolved, I'd need a way to > workaround the dhclient bug. > > Anybody have a clue why this is needed? > > Warner I have no clue, however if such needs are commonplace then maybe someone who's network-savvy can make a bug report. There's been tumbleweed, over the years, whenever I've tried to make things reliable for myself. Warner, I'll be interested to know whether what's below (also) works around things for you. Assuming ue0. route delete default ; ifconfig ue0 down && sleep 5 ; ifconfig ue0 up && sleep 15 ; ls /var/run/resolvconf/interfaces/ ; cat /etc/resolv.conf ; ping -c 2 -4 freshports.org From nobody Thu Aug 17 18:17:20 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRYBd2xbYz4mW1P for ; Thu, 17 Aug 2023 18:17:29 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRYBb0sqHz3P5v for ; Thu, 17 Aug 2023 18:17:27 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=BhisKB6I; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2a00:1450:4864:20::134 as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-4fe11652b64so24889e87.0 for ; Thu, 17 Aug 2023 11:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692296244; x=1692901044; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=ZFeY4DGnwH8Y9d7iFmebAjdXsIHUF7wvh4GkRtfjcpw=; b=BhisKB6IUAqEXrrqnY2c8c8Vona6xElue/RTJUbGZ0Z46zy907L1urxZaE8fg+JYds vo9jWQT+QA064nCwzmE4WDUVD3/RCl1GFHcA4XK69/fL8RrUYJh+1Fv5mgJGxHuZgM2M tkrmD9NlKfGGTYzqBz3DK15ZjRb7g8fei0UefGYXmfiFV177eLcgLt2KAPV+j08W0E0C 016DetOjo1PuX+sc8sG3JBcHkwB+LCUlPbm/i+UjrSHOquLyZkydfGJ3FuPRcAk1btYK wIqMieg5SbLVsWz1cImp8Nr+kMZGbwRAaEJtdR8rFFI+zuCSL4nlvJOzWKGOSW3RyeAG W8Zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692296244; x=1692901044; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=ZFeY4DGnwH8Y9d7iFmebAjdXsIHUF7wvh4GkRtfjcpw=; b=Wis7CcTurW2AKpR345BwgovyQcA8JGUWxqkNPxBz5mFiBDGGrRlkGoGeXFoPoVVntv oJFtUTKDbZ/BC6HU4V8xnGUTleBdHMBGsVhxMYuoFnSP/4kwy8GhDJ9OfGjTh1TDX3xD CEGIkjZ/q8EB0ak0POkP5fPu6MGJF7/0Tlqo137dGi5mdaV5mSoFy/65y/2rHXuvJ7Gg rykKjkwJlK9cSZ4/4MYxDutlZgjSFAHSg9hV+nqBdkrgVKMBLN5fifCcbIsmCd6Qo/KZ Tqu0WOdmI8VlcH/bCJunXMLPvwLJBDAybR6TUQUItDTelSTizAZIE36kcnnq9khP0Sg6 XrPQ== X-Gm-Message-State: AOJu0Yyrth9JCLOUlzZxhl0F0Y/WhdpqKUfLlDuBWoVobM1u+ccX2zAH j4aOxci4enZWZmJtBG8KgYf/7uaM3SA= X-Google-Smtp-Source: AGHT+IEDTWepzK8/x2+el+EaTXhwvtCqPRAxQpuD+XPa07ZGm406x+cwHkYKl8bQvsGv7fijdkw1Ug== X-Received: by 2002:ac2:4f16:0:b0:4fe:1b6c:7d37 with SMTP id k22-20020ac24f16000000b004fe1b6c7d37mr50626lfr.65.1692296243899; Thu, 17 Aug 2023 11:17:23 -0700 (PDT) Received: from [192.168.1.10] (80-42-66-93.dynamic.dsl.as9105.com. [80.42.66.93]) by smtp.gmail.com with ESMTPSA id gy26-20020a170906f25a00b0098e42bef732sm41937ejb.183.2023.08.17.11.17.23 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Aug 2023 11:17:23 -0700 (PDT) Message-ID: Date: Thu, 17 Aug 2023 19:17:20 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 To: freebsd-current@freebsd.org Content-Language: en-US From: Graham Perrin Subject: etcupdate -B, /.cshrc and /.profile Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[grahamperrin]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::134:from]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RRYBb0sqHz3P5v If I recall correctly, a few hours ago etcupdate -B resulted in removal of two files: /.cshrc /.profile Is this degree of checking/removal a novelty? (I can't recall the files' contents, or when I created them. I guess that I carelessly created them as dot files months ago without realising that I wasn't at ~, I don't mourn their loss.) Re: the manual pages below, I'm reminded of where files _should_ be. From nobody Thu Aug 17 18:57:31 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRZ5P01KKz4q1nL for ; Thu, 17 Aug 2023 18:58:00 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com [IPv6:2607:f8b0:4864:20::b32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRZ5M1xRwz3WGG; Thu, 17 Aug 2023 18:57:59 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=AimYHsDc; spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2607:f8b0:4864:20::b32 as permitted sender) smtp.mailfrom=mavbsd@gmail.com; dmarc=none Received: by mail-yb1-xb32.google.com with SMTP id 3f1490d57ef6-d66f105634eso166328276.1; Thu, 17 Aug 2023 11:57:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692298678; x=1692903478; h=content-transfer-encoding:in-reply-to:subject:from:content-language :references:cc:to:user-agent:mime-version:date:message-id:sender :from:to:cc:subject:date:message-id:reply-to; bh=Ro6sCefiCQUu/12q5Ri7oUSpPLgBhdMYdIPWQwjtctA=; b=AimYHsDcmd30rCZ8QzMFdQheWglJ/3NSJ3uxHiVC7fpzKUaEpS0hd4sAwNGAoW1xQO aOwEX5y46mmVxnskxF60kQ2R4H2TPP9i/ibSZaE1tDVvzdMAPi92i071HY1O6WvX6Bew y9puOyLnATaeG5VGAJWmlDSfaOn27EKrySZjjQcwfOluzu5AcZt6V0IvQHWejJlD5k+a 3jxT1ldKysxMWQbenXpm6zF5I4ty9/rmOQI1TXaSZcgsEG7z3OGirNLDXtU/t4A/mGAl R4cwI09kZJiaVeygT7f1+19AZcCjkxaaMPWXeHrgjliCY9CWqzSWupbwlOvZX/DiOQfO GwEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692298678; x=1692903478; h=content-transfer-encoding:in-reply-to:subject:from:content-language :references:cc:to:user-agent:mime-version:date:message-id:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ro6sCefiCQUu/12q5Ri7oUSpPLgBhdMYdIPWQwjtctA=; b=lE8qxJDD4+1lSIZPUsI4Xw+doPATxNaPi5mUr4rpvCVtYjDzXhOhD5LupsM8H8SHiU 7pS5uozUtavxObxh5wS3myj7sqAxTdCwvBa5YXU0xpiTrNRplerLxtyjgL5j4q4C6m1W 1xm/eqRU5aiIH1rfEz1uPl4fIrcqnNRoDiZ/VIefl2Dj+QFtCxujHKx70X2k1fO0knDb DYRe3sACjvgYocgxJYotTq0iwaSb+8GRz7ogLIHyoqdBjyll+J4HdZ1dQeaCvoLUbNHo 50G7KutzTVXQJLaJHy996qE5yG7wB3r6qEYLvbxVzL/rKPAncWnT9RlLYti+o/p0llNh oypw== X-Gm-Message-State: AOJu0YyEiylXmOBnfTzUL35Lx5G7LoNcHGyAOlTeQjwts9lOikh9Yfd+ 7bj1VJc7IuPuhR+/vdQhpCg+JW28T+s= X-Google-Smtp-Source: AGHT+IFKRQhbHx5VuGyhZ+lBB8gcfy+KSteNUMK24s3d6TSTIsJd+GATWEceqWNDmIqNSeCKUPvL8g== X-Received: by 2002:a25:3615:0:b0:d32:cd49:2469 with SMTP id d21-20020a253615000000b00d32cd492469mr459811yba.24.1692298677912; Thu, 17 Aug 2023 11:57:57 -0700 (PDT) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id 193-20020a2502ca000000b00d71def6d29fsm25195ybc.12.2023.08.17.11.57.56 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Aug 2023 11:57:57 -0700 (PDT) Message-ID: <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> Date: Thu, 17 Aug 2023 14:57:31 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 To: =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> Content-Language: en-US From: Alexander Motin Subject: Re: ZFS deadlock in 14 In-Reply-To: <86ttt0p8wv.fsf@ltc.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.20 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b32:from]; DMARC_NA(0.00)[freebsd.org]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com,FreeBSD.org]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RRZ5M1xRwz3WGG On 15.08.2023 12:28, Dag-Erling Smørgrav wrote: > Mateusz Guzik writes: >> Going through the list may or may not reveal other threads doing >> something in the area and it very well may be they are deadlocked, >> which then results in other processes hanging on them. >> >> Just like in your case the process reported as hung is a random victim >> and whatever the real culprit is deeper. > > We already know the real culprit, see upthread. Dag, I looked through the thread once more, and, while thank you for tracing it, but you never went beyond txg_wait_synced() in `zfs revert` thread. If you are saying that thread is holding the lock, then the question is why transaction commit is stuck. I need to see stacks for ZFS sync threads, or better all kernel stacks, just in case. Without that information I can only speculate. Trying to run your test (so far without reproduction) I see it producing a substantial amount of ZIL writes. The range of commits you reduced the scope to so far includes my ZIL locking refactoring, where I know for sure are some deadlocks. I am already waiting for 3 weeks now for reviews and tests for PR that should fix it: https://github.com/openzfs/zfs/pull/15122 . It would be good if you could test it, though it seems to depend on few more earlier patches not merged to FreeBSD yet. -- Alexander Motin From nobody Thu Aug 17 19:32:12 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRZs45hZGz4q44d for ; Thu, 17 Aug 2023 19:32:24 +0000 (UTC) (envelope-from sr@genyosha.net) Received: from ns0.genyosha.net (ns0.genyosha.net [50.39.243.220]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "float.home.genyosha.net", Issuer "float.home.genyosha.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRZs436gmz3ZCc for ; Thu, 17 Aug 2023 19:32:24 +0000 (UTC) (envelope-from sr@genyosha.net) Authentication-Results: mx1.freebsd.org; none Received: from dragon.home.genyosha.net (ops0.genyosha.net [50.39.243.219]) by ns0.genyosha.net (8.17.1/8.17.1) with ESMTPS id 37HJWH1r067640 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 17 Aug 2023 12:32:17 -0700 (PDT) (envelope-from sr@genyosha.net) Received: from dragon.home.genyosha.net (localhost [127.0.0.1]) by dragon.home.genyosha.net (8.14.7/8.14.7) with ESMTP id 37HJWC4D025867; Thu, 17 Aug 2023 12:32:12 -0700 Received: (from sr@localhost) by dragon.home.genyosha.net (8.14.7/8.14.7/Submit) id 37HJWCdA025866; Thu, 17 Aug 2023 12:32:12 -0700 Date: Thu, 17 Aug 2023 12:32:12 -0700 From: Steve Rikli To: Graham Perrin Cc: freebsd-current@freebsd.org Subject: Re: etcupdate -B, /.cshrc and /.profile Message-ID: References: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Greylist: inspected by milter-greylist-4.6.4 (ns0.genyosha.net [50.39.243.220]); Thu, 17 Aug 2023 12:32:17 -0700 (PDT) for IP:'50.39.243.219' DOMAIN:'ops0.genyosha.net' HELO:'dragon.home.genyosha.net' FROM:'sr@genyosha.net' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (ns0.genyosha.net [50.39.243.220]); Thu, 17 Aug 2023 12:32:17 -0700 (PDT) X-Rspamd-Queue-Id: 4RRZs436gmz3ZCc X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:27017, ipnet:50.39.128.0/17, country:US] On Thu, Aug 17, 2023 at 07:17:20PM +0100, Graham Perrin wrote: > If I recall correctly, a few hours ago etcupdate -B resulted in removal of > two files: > > /.cshrc > /.profile > > Is this degree of checking/removal a novelty? > > (I can't recall the files' contents, or when I created them. I guess that I > carelessly created them as dot files months ago without realising that I > wasn't at ~, I don't mourn their loss.) I suspect you didn't create them, at least not directly or manually; my system still has them at the moment, and they appear to be the same from /root/ : $ uname -mrv 13.2-STABLE FreeBSD 13.2-STABLE stable/13-n255938-fec383bb5385 GENERIC amd64 $ ls -lai /.profile /root/.profile /.cshrc /root/.cshrc 5 -rw-r--r-- 2 root wheel 1023 May 12 2022 /.cshrc 7 -rw-r--r-- 2 root wheel 507 May 12 2022 /.profile 5 -rw-r--r-- 2 root wheel 1023 May 12 2022 /root/.cshrc 7 -rw-r--r-- 2 root wheel 507 May 12 2022 /root/.profile I think they were installed by FreeBSD rather than created. My other systems which have been periodically upgraded over time have the same situation. Cheers, sr. From nobody Thu Aug 17 19:33:33 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRZtX4TdBz4q453 for ; Thu, 17 Aug 2023 19:33:40 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRZtX1Xsvz3bDW; Thu, 17 Aug 2023 19:33:40 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Authentication-Results: mx1.freebsd.org; none Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 7CFFC3C0199; Thu, 17 Aug 2023 19:33:33 +0000 (UTC) Date: Thu, 17 Aug 2023 19:33:33 +0000 From: Brooks Davis To: Ronald Klop Cc: freebsd-current@freebsd.org Subject: Re: building with llvm16 pkg fails in tests Message-ID: References: <76ecade1-3f42-a314-825d-6f1ba0b2dcaa@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <76ecade1-3f42-a314-825d-6f1ba0b2dcaa@FreeBSD.org> X-Rspamd-Queue-Id: 4RRZtX1Xsvz3bDW X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US] On Thu, Aug 17, 2023 at 12:45:06PM +0200, Ronald Klop wrote: > Hi, >=20 > To save time on my Raspberry Pi I would like to build FreeBSD using a llv= m pkg instead of llvm in the tree. >=20 > My /etc/make.conf: > WITHOUT_TOOLCHAIN=3Dyes > LD=3D/usr/local/llvm16/bin/ld.lld > CC=3D/usr/local/llvm16/bin/clang > CXX=3D/usr/local/llvm16/bin/clang++ > CPP=3D/usr/local/llvm16/bin/clang-cpp > OBJCOPY=3D/usr/local/llvm16/bin/llvm-objcopy >=20 > #WITHOUT_CLEAN=3Dyes >=20 >=20 > This fails in: >=20 > /usr/local/llvm16/bin/clang++ -O2 -pipe -fno-common -fPIE -Wno-format-zer= o-length -nobuiltininc -idirafter /usr/local/llvm16/lib/clang/16/include -f= stack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -W= no-unused-parameter -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strin= gs -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Wda= te-time -Wmissing-variable-declarations -Wno-empty-body -Wno-string-plus-in= t -Wno-unused-const-variable -Wno-error=3Dunused-but-set-parameter -O0 -g0 = -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/usr/include/private -= I/usr/src/contrib/googletest/googlemock/include -I/usr/src/contrib/googlete= st/googlemock -I/usr/src/contrib/googletest/googletest/include -I/usr/src/c= ontrib/googletest/googletest -I/usr/obj/usr/src/amd64.amd64/tmp/usr/include= /private -DGTEST_HAS_POSIX_RE=3D1 -DGTEST_HAS_PTHREAD=3D1 -DGTEST_HAS_STREA= M_REDIRECTION=3D1 -frtti -Wno-deprecated-copy -Wno-signed-unsigned-wchar -D= GTEST_HAS_POSIX_RE=3D1 -DGTEST_HAS_PTHREAD=3D1 -DGTEST_HAS_STREAM_REDIRECTI= ON=3D1 -frtti -Wno-deprecated-copy -Wno-signed-unsigned-wchar -fPIE -std=3D= c++14 -Wno-deprecated-copy -Wno-error=3Dinconsistent-missing-override -Wno-= error=3Dmissing-variable-declarations -Wno-error=3Dsign-compare -Wno-error= =3Dunused-parameter -Wno-c++11-extensions -Wl,-zrelro -pie --ld-path=3D/u= sr/local/llvm16/bin/ld.lld -o gmock-actions_test gmock-actions_test.o -lpr= ivategmock_main -lprivategmock -lprivategtest > ld.lld: error: undefined symbol: testing::internal::DeathTest::Create(cha= r const*, testing::Matcher, std::__1::allocator> const&>, char const*, int, testing::in= ternal::DeathTest**) > >>> referenced by gmock-actions_test.cc > >>> gmock-actions_test.o:(testing::(anonymous namespace)::B= uiltInDefaultValueDeathTest_IsUndefinedForReferences_Test::TestBody()) > >>> referenced by gmock-actions_test.cc > >>> gmock-actions_test.o:(testing::(anonymous namespace)::B= uiltInDefaultValueDeathTest_IsUndefinedForReferences_Test::TestBody()) > >>> referenced by gmock-actions_test.cc > >>> gmock-actions_test.o:(testing::(anonymous namespace)::B= uiltInDefaultValueDeathTest_IsUndefinedForNonDefaultConstructibleType_Test:= :TestBody()) > >>> referenced 4 more times >=20 > ld.lld: error: undefined symbol: testing::Expectation::Expectation(std::_= _1::shared_ptr const&) >=20 >=20 > Any thoughts on how to fix this? > Compiling with the in tree llvm does work properly. Did it work with exactly this git revision? I suspect an issue with the recent google test update rather than an llvm16 issue. Note that for every sync to github we build the tree with the llvm16 port (all be it on amd64 by default). It's worth noting one difference between your configuration and the CI one: We don't set CC and friends directly. Instead we use CROSS_TOOLCHAIN=3Dllvm16. -- Brooks From nobody Thu Aug 17 19:37:09 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRZz71kNKz4q3lT for ; Thu, 17 Aug 2023 19:37:39 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-oi1-x22b.google.com (mail-oi1-x22b.google.com [IPv6:2607:f8b0:4864:20::22b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRZz56nCjz3cFX; Thu, 17 Aug 2023 19:37:37 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=Vpe6s2+a; spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2607:f8b0:4864:20::22b as permitted sender) smtp.mailfrom=mavbsd@gmail.com; dmarc=none Received: by mail-oi1-x22b.google.com with SMTP id 5614622812f47-3a82db15081so92995b6e.0; Thu, 17 Aug 2023 12:37:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692301056; x=1692905856; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=i3SZLJjPaeNIYyJ7S9T84pHP4E+l/AOnCeWTCdfZ0r4=; b=Vpe6s2+ag0OLmv4reTbxHJFZnN4/6GD4SuKRmfYpvaiwXW1zlQtVBR+u4ufgFwE7Qh /tIX8pHPP1J79sW0B7ke0pX3rkE+lgT3u9CzEeWmJwnrYLhRXo+i8MQlta3CLne3zUXU sVlCl3QLBe5x73Tv9IZrCQ6VDDXVyLSMEn7RY3QRYhh7WnlUisyEi2Xk87DdvnUEwSWQ TMPZB+stY7WqdrwkLFXQGQM2k2f4s3dgO5TnFa1u1VKCzbNkzP29fbvY3ishnUmXg4y4 NcXglqdMl9PkA4UcCFkw/TzdnvuKyNNB1lQXQbFCazVJ0HJk488En9qERNO0dBND4I7w sjbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692301056; x=1692905856; h=content-transfer-encoding:in-reply-to:references:cc:to:from :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=i3SZLJjPaeNIYyJ7S9T84pHP4E+l/AOnCeWTCdfZ0r4=; b=Ubcp0WwZksTGIf1eyjv30J3rMnbBP12iugmx0gh1ZOJyyMcIDW/yjjTPPCHM+50bpu Las839UpklnIXa4bt5Vz5cuk7cYyALMx8CrxxkKXVcxyq2/fVscfb7gtTVSpVEp2t7Jy Qm0trZ31RDVxi0JDC6u5Iw4sXJrCqW4mJp7FOblU/zfPtIGMyhRIUNkqOzFM5nRvZYj6 LL/SfGOVULv0MrmTKXb14CUI6kZDXz7mIpTSMTwCTko9bbOr4xengjO6ZDek5700h//K NOaMEpPbFs6SbyRKynY0cmRsRkDg/QpLRz56mc5Y3eHr2XbTUjViDBf1Z5xroUPFQ6F/ CcDQ== X-Gm-Message-State: AOJu0YyCrW9KbpoAAUNx3AW7PvTfxDrtGsHqox+8kzjz7xwxRCY+I1Gm apRopJ1bCyucIewhsPlIhKKeBMEeXHY= X-Google-Smtp-Source: AGHT+IEv8FJKrtXG40Erg/W5xpzlqpjHjMgO6W+ebE2JgQG1zCYfmc8lOYDw6ljm3fKGxD3s9Jc5ww== X-Received: by 2002:a05:6358:6f16:b0:135:499b:a68c with SMTP id r22-20020a0563586f1600b00135499ba68cmr394666rwn.8.1692301055690; Thu, 17 Aug 2023 12:37:35 -0700 (PDT) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id z1-20020a25ad81000000b00c5fc63686f1sm41889ybi.16.2023.08.17.12.37.34 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Aug 2023 12:37:35 -0700 (PDT) Message-ID: <8c88acdc-7009-9801-ef44-3e1359c59aff@FreeBSD.org> Date: Thu, 17 Aug 2023 15:37:09 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: ZFS deadlock in 14 Content-Language: en-US From: Alexander Motin To: =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> In-Reply-To: <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.16 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.964]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22b:from]; DMARC_NA(0.00)[freebsd.org]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com,FreeBSD.org]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RRZz56nCjz3cFX On 17.08.2023 14:57, Alexander Motin wrote: > On 15.08.2023 12:28, Dag-Erling Smørgrav wrote: >> Mateusz Guzik writes: >>> Going through the list may or may not reveal other threads doing >>> something in the area and it very well may be they are deadlocked, >>> which then results in other processes hanging on them. >>> >>> Just like in your case the process reported as hung is a random victim >>> and whatever the real culprit is deeper. >> >> We already know the real culprit, see upthread. > > Dag, I looked through the thread once more, and, while thank you for > tracing it, but you never went beyond txg_wait_synced() in `zfs revert` > thread.  If you are saying that thread is holding the lock, then the > question is why transaction commit is stuck.  I need to see stacks for > ZFS sync threads, or better all kernel stacks, just in case.  Without > that information I can only speculate. > > Trying to run your test (so far without reproduction) I see it producing > a substantial amount of ZIL writes.  The range of commits you reduced > the scope to so far includes my ZIL locking refactoring, where I know > for sure are some deadlocks.  I am already waiting for 3 weeks now for > reviews and tests for PR that should fix it: > https://github.com/openzfs/zfs/pull/15122 .  It would be good if you > could test it, though it seems to depend on few more earlier patches not > merged to FreeBSD yet. Ah, appears on the pool I tested first I have sync=always from earlier tests, that explains the high amount of ZIL traffic I saw, so it may be irrelevant. But I still wonder what sync threads are doing in your case. -- Alexander Motin From nobody Thu Aug 17 19:41:37 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRb3n1rZwz4q4Mg for ; Thu, 17 Aug 2023 19:41:41 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRb3m6s5gz3fsg; Thu, 17 Aug 2023 19:41:40 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692301301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h6XFIZEgOBddXcJy4MG458BAEGVeFevtfMrQD7/MEOc=; b=J6EfZhGQbcZXWykS8pe4XTYng2sUNEE33+LeZuvfxrKefQUn01Rur57Sx5XLgh4YYxyfmi xMK3tAwb1PF2YxicC90/GV8cPMjjAQvZSzZI24kmtVZ/BMobg7rhG3Y0K+GA4yTCnaWe69 vu10TCPSMiikI7WFE+WzMtX6d9gx2PxD8Nz63bibtxSYA2Jbvxj+CW3DfGxoqwkqdEyYrK t2dmsdHXScPF6Qj25yd0N84FlGaN9/f0vdSG7gHv5nhJMUGnh5Xm0XWiVrrXvbPJizd0Nh /BbpRBhjA7rqZ7os/yWTYaZojfEBehIaTFhbOAB5aJECCo0jHiX8kqCrKsOViA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692301301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h6XFIZEgOBddXcJy4MG458BAEGVeFevtfMrQD7/MEOc=; b=Ysw8rUClztmtf3xAz0+xAIr/xVnGK7xolCz70mEhXG5vEx1BZkfEdCLBrdQS9oAsU10cc0 /VaejSTwqfu1i3uDzuu1fRdYtGPucgliKjTlfHTaVXJvYoBD6bXl/e759Evl93A6PHSUTb e6gZGIfhA6ztS6P0rWoncxjz8THRKtCgi2TXxC2ZNTHFuRFf0ngY/RbnGPkOS6m62R1BJU kT+HNVvGxXpBEm04bUDnPtVp5oiwbdq0kr40kOjZpdMbnhEXnGD+r9BH5tLSa+Rkbu8KOH 5YhegpU8MJMhQNtdzHzvqF7JLFj594hNZKSjRNzSfqV/i6LnFP2BH10lKcfuDA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692301301; a=rsa-sha256; cv=none; b=vhbUFLHkjk1q6l8e6Y9961VavBvezLlnlIo7GpVY6VK75W7ya9z4FdYZsqPUw6GzQRqJDM VqyFiCWfFFBoaC5HWcpzoaGqo7iYkszluI9d8cZB2LDKelhAjeVeh+swTuBMVNdFai/zpM tgMNiEmMJxTnSEBfMxYRQgzhKmyMG8xBVBEtHmIUEPGZZN24XrVPcmTqorsfnxaOrbiiM1 d+naHaVKA+cm3a3J3BZy+3BOINgFMJTMzmdI8c38l5NsOOcR4iZbcmU6E642I+YkjR8Sf3 pj0xSwV9HJw28u/g22SlAUrUgHgSYmD8xPMHvPCjNLExTWLrrnenAPqjSR3AfQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (unknown [IPv6:2001:4647:d671:0:36e8:94ff:feca:9834]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RRb3m5D2rztP3; Thu, 17 Aug 2023 19:41:40 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id D7F4318268; Thu, 17 Aug 2023 21:41:37 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Alexander Motin Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska Subject: Re: ZFS deadlock in 14 In-Reply-To: <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> (Alexander Motin's message of "Thu, 17 Aug 2023 14:57:31 -0400") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Thu, 17 Aug 2023 21:41:37 +0200 Message-ID: <86350hpiby.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Alexander Motin writes: > Dag, That's not my name, Al. > Trying to run your test (so far without reproduction) I see it > producing a substantial amount of ZIL writes. The range of commits > you reduced the scope to so far includes my ZIL locking refactoring, > where I know for sure are some deadlocks. I am already waiting for 3 > weeks now for reviews and tests for PR that should fix it: > https://github.com/openzfs/zfs/pull/15122 . It would be good if you > could test it, though it seems to depend on few more earlier patches > not merged to FreeBSD yet. Do you have a FreeBSD branch with your patch applied? DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Thu Aug 17 19:46:25 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRb9b3twKz4q4Ly for ; Thu, 17 Aug 2023 19:46:43 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-24.consmr.mail.gq1.yahoo.com (sonic311-24.consmr.mail.gq1.yahoo.com [98.137.65.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRb9Y4s8cz4G0b for ; Thu, 17 Aug 2023 19:46:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=bhikjPiQ; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692301599; bh=sF9kwWkHqxye8KQUr174M9ElGzum7aaEfHn2czAEEHA=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=bhikjPiQsnSELj0R+kX+P6KzIAoW0lb4rQQPJJSUnp0OVH1LTWkrM+yY0flmcCphATSDwcLdPRGugq1w33oknL/O9+i8scJddbd667UmUkSnQ07Ud0CkRvZVs4iJy7mp2AnFN4IsgREgqQDhPsha6ymOx7lxeydJtaodqysrpdhyUB664ZoU2wbDgA3lCOVXS/xl56FIwtZPYnVO7Qn07N2GPSa6MoKy8H8yIYzB5+XOlLEPmMWfa8bSn7lwqm5vnHBa8HhSOoXp32XR6otpAd3uDjvJmRCiV1KiDVnO1HvAunO4mPmuyFpMQQZqhTgOeo21pd8JRjZ9TMvzQZ2URw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692301599; bh=ph/S5GDtFM1NH6KXhtbaxWUigEB9aald0K4s5pb5umX=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=CBBc7arIXBYEuVLDQx1Apy4JzL/OVOp/ckQCniwRid7IqZYSlweU1HlYEWZkhDwWRqXXQsSdDoeDspY6BenNOu7oBgdxOO+in6qWxy9CqKOaNuzs+6T1nCx/Jx9EnzHnJooGjPICY5LATO2tdJjFZolY0kuINh99PKKnhea5gNe13ivrPFVsBLvdvtGID0zr1FfqIPErXMBPWV+J2Jx6XcVUVbGZ/o5YY8aGJbCtbycY0ORtrrK5A7Y6s+VPLUiMyprk0ytWE4SQJ+uNX/v6XKV6S9HcKpI3BQb2UVuBYLUubmetC2t/rKfVpQZaeB6TYpdVWfTibSOUN1AIqjBodg== X-YMail-OSG: lBXipeIVM1lKaeY0cuKpxX9tzaakoRED6f1.2C1.uPqm3GLg2.62WNLrO0WT4wb WuY2wPFR4S0J9yT4NKYP4rce9pexKugpemJFNAnLkmMgJrKhYX8lfuYGn685zinJx8.Lka13nTWF 8RoKX0QJ3PNoDmDq6QXrpuna0VAyyFMOyoBzEqvg3_eCFRumTZwpk750a6FSS6IQPDhqLsHD68.h Yr3vpkLuFUD13.w.09Dl2wUn6yz4jMGZ9TsMihsauhg4ltGmsTMWDFBfDlgqXNRbbX0FKXXVAHhS qTuj6fX7PH8rP4f9c7ILwUorje9Jkz1wRA4QehplVsfRJkgBL2PG1Pqo21WaPyMTQkOVahhue7e2 1KGfDrBeVvpFxvX3A8K9uDLa6SXP.C9NVd8xRJ0eNczaO_LPlsTUOiZ21ponQ.c0hP40eMwLBozX Fou0m7rLf4Dr78h54g3o8MfHhmvSeZxqS4E5pOZK6yMyjWwjsq_7gvo_gkyRYtzz6YXnB6lt2TT7 _wZ_yLFFipuDhmvgt7JzQnFMVgFbxOMI_siHHswfyU2DRno4gtmkUUJYQtNR55Hadh_wJ1IqX66W zyQwDGt0fdOIhlIwqfyR2FAP.6LgJ2N1W6GibCBzHcRuSzro70FJYlW.Ga8F29yIPrlEXAE9vvq2 piIREHdElACef5MO2C6N_sXBAF3m8lpq0eInJcDdHji2QhzdCydK_0083rVSsNV25InjOfOlPct5 R7y4wTE0s45jKGG6Qm0bfxfsoXPBzfIfmoHixlRvf7WtuA__PyjhKxpN2mYdHaqbd.irrrQTaO42 6z7hkNqNTWC_ykGkB5B8HmJvHJUxOGIhNlRz7SRM8La3qiHEZKGoJ2ERU.wMq.teqJNAkXg.2L8J UQxQmZRuc.iKrvZU2wRu5H9wYr85V989V0eC1UTI0bPhkA9oEZtJ6wK.k1MdUf1Trdv4MCWnUUs5 AzPsMqP4xu.j5S2IjgyG.31SWvltWPz9Y8a8oyVZTry1PxZWRqtTDTulH28l7fK23aGhNAWRFJuq 4VrTuQgXDFSz_MVy189sHYmIcF3vyn4GHvETN4BHVlxl848KS8Eb5asrQ2f9cJy.qb5PkFvh9VL0 Et5jK_27yhrKtemSSyrxZuRv34lc.tyDKJeDjiCfMPTokcL4Yd2SnPRgSXKBkxKvoX48XQc7713X n80QHwca2NjR9V0WL6JBcSTuIydzOht._Ntz1wkUHGv.vLdBDTTFHPKxbtEo.2FCrvMslkUgRddg EXKK9fO3HWlY7LWx6t0PWJZVLPM0iAlEKRImKYkpBiMX9ghiU6xRQFefV4HI9Mba9Jt0MLPRfrmc xOG2T7zEV.ohc7l76cOb2prla9yjItRv7umPHg2l6VdVXe.rJ46owJiJQ1YwNapmEb1_rSjeHV65 cJaSwoSPOfmEMSuWA2DXCUVJ7arnpUxvf.y9zTj_n_Ex9FpDVCMtjjdGo6C1oE4Y6rtSoc5TXEQb UgTi55iRxRRCFBd4.sXQ5DHu_uZ7OWj2pnwqXJGf4Gx5f2rTXx2Qe127Ij2rliRTAzSP3f0VPhRW QXaqWu7tCkiGS.Q34wohPviz4wyTEi6PJbd46iSjrU6BJ.L.U0.E6_KB3iXAnHByEzAnikCMGXbl bJIKfHC3GVnLCRaA4hJ_V6HUsGWcNWkXBb4Y.0F_Wa06gRfBwUbUhg1BISS19pO47tv6WMqY53nu tx0HN9WQs0LYK6k9VQHBix358SdeCgw0.idr8Ec1IjuZzMVmb8.GX7_.Yb96EE1VK5tOYNF_9T3E GH6kyr.THPPDKUuG3a6yHYEHkQp8Z2b1Fm45WA5fBAX1QV79mSuu_Fe0XADy9J0HCehnRzMxLEQ7 hOABYNkxYDxIsDJPFXni7yrtxgcX3Eu0hVuqZ.PrdFMe0W.70fCuFpgwt4CUcGSG2X9HVlg2XwpR VC5IXGvogOxgGirv9twdvqhrlaqyByal6RqaFJuixkztdvJzY1_8RyAeR17mkornwxGtA0RLZPns UdE3m4AnkXySlRleq1vdmuPq2PHM3ivPR1wxvGMYkuy3yvLQDbOe849C_l.ZAONWpqePi4WoEXsm G6HV18ZSzvHU4d6G8C6AdK8VbV07ZVNzDmW67UK6o0BoxhLx1vnMGGHw3LlYPd9pdZ0Uxq8_6G6i Yec9X.S99.GPJg1IH.evaZ6rH1Uvk65kye5Gy4HaSoW5uBB8dTfgoHlTjdrrQlA5_nl7_Vu9aVN0 VZ1aW4PDD6muYMbE837MiwiaNrQWau8LB0y3Ii1XeIYwB6KtGpg1f6lf2c_MAMXRylDlO7vukmuU iRQ-- X-Sonic-MF: X-Sonic-ID: b89b20e1-e302-46a7-a1b6-2f0cdec33268 Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Thu, 17 Aug 2023 19:46:39 +0000 Received: by hermes--production-gq1-6b7c87dcf5-wb6qb (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 10fe418b02a94fc944a856818b9330fd; Thu, 17 Aug 2023 19:46:36 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: building with llvm16 pkg fails in tests Message-Id: Date: Thu, 17 Aug 2023 12:46:25 -0700 To: Ronald Klop , Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.205:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.205:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RRb9Y4s8cz4G0b Ronald Klop wrote on Date: Thu, 17 Aug 2023 10:45:06 UTC : > To save time on my Raspberry Pi I would like to build FreeBSD using a = llvm pkg instead of llvm in the tree. >=20 > My /etc/make.conf: > WITHOUT_TOOLCHAIN=3Dyes > LD=3D/usr/local/llvm16/bin/ld.lld > CC=3D/usr/local/llvm16/bin/clang > CXX=3D/usr/local/llvm16/bin/clang++ > CPP=3D/usr/local/llvm16/bin/clang-cpp > OBJCOPY=3D/usr/local/llvm16/bin/llvm-objcopy >=20 > #WITHOUT_CLEAN=3Dyes More normal would likely be something like (but expressed just in command line notation below): make -j$(sysctl -n hw.ncpu) CROSS_TOOLCHAIN=3Dllvm16 = WITHOUT_TOOLCHAIN=3Dyes buildworld buildkernel The CROSS_TOOLCHAIN hook is already present and avoids needing the explicit assignments to LD, CC, CXX, CPP, and OBJCOPY . (Those presume the host is compatible with the target, by the way. See below for notation that does not presume such --but allows such.) For reference, CROSS_TOOLCHAIN=3Dllvm16 uses: # more /usr/local/share/toolchains/llvm16.mk XCC=3D/usr/local/bin/clang16 XCXX=3D/usr/local/bin/clang++16 XCPP=3D/usr/local/bin/clang-cpp16 XLD=3D/usr/local/bin/ld.lld16 CROSS_BINUTILS_PREFIX=3D/var/empty X_COMPILER_TYPE=3Dclang that is installed as part of devel/llvm16 . > This fails in: >=20 > /usr/local/llvm16/bin/clang++ -O2 -pipe -fno-common -fPIE = -Wno-format-zero-length -nobuiltininc -idirafter = /usr/local/llvm16/lib/clang/16/include -fstack-protector-strong = -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter = -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-strings -Wswitch = -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -Wdate-time = -Wmissing-variable-declarations -Wno-empty-body -Wno-string-plus-int = -Wno-unused-const-variable -Wno-error=3Dunused-but-set-parameter -O0 -g0 = -Qunused-arguments = -I/usr/obj/usr/src/amd64.amd64/tmp/usr/include/private = -I/usr/src/contrib/googletest/googlemock/include = -I/usr/src/contrib/googletest/googlemock = -I/usr/src/contrib/googletest/googletest/include = -I/usr/src/contrib/googletest/googletest = -I/usr/obj/usr/src/amd64.amd64/tmp/usr/include/private = -DGTEST_HAS_POSIX_RE=3D1 -DGTEST_HAS_PTHREAD=3D1 = -DGTEST_HAS_STREAM_REDIRECTION=3D1 -frtti -Wno-deprecated-copy = -Wno-signed-unsigned-wchar -DGTEST_HAS_POSIX_RE=3D1 = -DGTEST_HAS_PTHREAD=3D1 -DGTEST_HAS_STREAM_REDIRECTION=3D1 -frtti = -Wno-deprecated-copy -Wno-signed-unsigned-wchar -fPIE -std=3Dc++14 = -Wno-deprecated-copy -Wno-error=3Dinconsistent-missing-override = -Wno-error=3Dmissing-variable-declarations -Wno-error=3Dsign-compare = -Wno-error=3Dunused-parameter -Wno-c++11-extensions -Wl,-zrelro -pie = --ld-path=3D/usr/local/llvm16/bin/ld.lld -o gmock-actions_test = gmock-actions_test.o -lprivategmock_main -lprivategmock -lprivategtest > ld.lld: error: undefined symbol: = testing::internal::DeathTest::Create(char const*, = testing::Matcher, std::__1::allocator> const&>, char = const*, int, testing::internal::DeathTest**) > >>> referenced by gmock-actions_test.cc > >>> gmock-actions_test.o:(testing::(anonymous = namespace)::BuiltInDefaultValueDeathTest_IsUndefinedForReferences_Test::Te= stBody()) > >>> referenced by gmock-actions_test.cc > >>> gmock-actions_test.o:(testing::(anonymous = namespace)::BuiltInDefaultValueDeathTest_IsUndefinedForReferences_Test::Te= stBody()) > >>> referenced by gmock-actions_test.cc > >>> gmock-actions_test.o:(testing::(anonymous = namespace)::BuiltInDefaultValueDeathTest_IsUndefinedForNonDefaultConstruct= ibleType_Test::TestBody()) > >>> referenced 4 more times >=20 > ld.lld: error: undefined symbol: = testing::Expectation::Expectation(std::__1::shared_ptr const&) I'm not making claims about the specifics of this error being produced vs. not the "more normal" way. But it would be worth a try. > Any thoughts on how to fix this? > Compiling with the in tree llvm does work properly. >=20 > NB: building 14-CURRENT in a 14-CURRENT jail on 13.2-RELEASE. And this = example is my test on amd64 to make it work before I will do this in the = slow RPI4. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Aug 17 20:00:10 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRbT838FZz4q55P for ; Thu, 17 Aug 2023 20:00:12 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRbT70w3jz4L2X; Thu, 17 Aug 2023 20:00:11 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of brooks@spindle.one-eyed-alien.net has no SPF policy when checking 199.48.129.229) smtp.mailfrom=brooks@spindle.one-eyed-alien.net; dmarc=none Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id B12B93C0199; Thu, 17 Aug 2023 20:00:10 +0000 (UTC) Date: Thu, 17 Aug 2023 20:00:10 +0000 From: Brooks Davis To: Ronald Klop Cc: freebsd-current@freebsd.org Subject: Re: building with llvm16 pkg fails in tests Message-ID: References: <76ecade1-3f42-a314-825d-6f1ba0b2dcaa@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: X-Spamd-Result: default: False [-1.58 / 15.00]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.88)[-0.885]; FORGED_SENDER(0.30)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; TO_DN_SOME(0.00)[]; R_SPF_NA(0.00)[no SPF record]; ARC_NA(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; RCPT_COUNT_TWO(0.00)[2]; DMARC_NA(0.00)[freebsd.org]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[brooks]; FROM_NEQ_ENVFROM(0.00)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; RCVD_TLS_LAST(0.00)[] X-Spamd-Bar: - X-Rspamd-Queue-Id: 4RRbT70w3jz4L2X On Thu, Aug 17, 2023 at 07:33:33PM +0000, Brooks Davis wrote: > On Thu, Aug 17, 2023 at 12:45:06PM +0200, Ronald Klop wrote: > > Hi, > >=20 > > To save time on my Raspberry Pi I would like to build FreeBSD using a l= lvm pkg instead of llvm in the tree. > >=20 > > My /etc/make.conf: > > WITHOUT_TOOLCHAIN=3Dyes > > LD=3D/usr/local/llvm16/bin/ld.lld > > CC=3D/usr/local/llvm16/bin/clang > > CXX=3D/usr/local/llvm16/bin/clang++ > > CPP=3D/usr/local/llvm16/bin/clang-cpp > > OBJCOPY=3D/usr/local/llvm16/bin/llvm-objcopy > >=20 > > #WITHOUT_CLEAN=3Dyes > >=20 > >=20 > > This fails in: > >=20 > > /usr/local/llvm16/bin/clang++ -O2 -pipe -fno-common -fPIE -Wno-format-z= ero-length -nobuiltininc -idirafter /usr/local/llvm16/lib/clang/16/include = -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -W = -Wno-unused-parameter -Wpointer-arith -Wreturn-type -Wcast-qual -Wwrite-str= ings -Wswitch -Wshadow -Wunused-parameter -Wcast-align -Wchar-subscripts -W= date-time -Wmissing-variable-declarations -Wno-empty-body -Wno-string-plus-= int -Wno-unused-const-variable -Wno-error=3Dunused-but-set-parameter -O0 -g= 0 -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/usr/include/private= -I/usr/src/contrib/googletest/googlemock/include -I/usr/src/contrib/google= test/googlemock -I/usr/src/contrib/googletest/googletest/include -I/usr/src= /contrib/googletest/googletest -I/usr/obj/usr/src/amd64.amd64/tmp/usr/inclu= de/private -DGTEST_HAS_POSIX_RE=3D1 -DGTEST_HAS_PTHREAD=3D1 -DGTEST_HAS_STR= EAM_REDIRECTION=3D1 -frtti -Wno-deprecated-copy -Wno-signed-unsigned-wchar = -DGTEST_HAS_POSIX_RE=3D1 -DGTEST_HAS_PTHREAD=3D1 -DGTEST_HAS_STREAM_REDIREC= TION=3D1 -frtti -Wno-deprecated-copy -Wno-signed-unsigned-wchar -fPIE -std= =3Dc++14 -Wno-deprecated-copy -Wno-error=3Dinconsistent-missing-override -W= no-error=3Dmissing-variable-declarations -Wno-error=3Dsign-compare -Wno-err= or=3Dunused-parameter -Wno-c++11-extensions -Wl,-zrelro -pie --ld-path=3D= /usr/local/llvm16/bin/ld.lld -o gmock-actions_test gmock-actions_test.o -l= privategmock_main -lprivategmock -lprivategtest > > ld.lld: error: undefined symbol: testing::internal::DeathTest::Create(c= har const*, testing::Matcher, std::__1::allocator> const&>, char const*, int, testing::= internal::DeathTest**) > > >>> referenced by gmock-actions_test.cc > > >>> gmock-actions_test.o:(testing::(anonymous namespace):= :BuiltInDefaultValueDeathTest_IsUndefinedForReferences_Test::TestBody()) > > >>> referenced by gmock-actions_test.cc > > >>> gmock-actions_test.o:(testing::(anonymous namespace):= :BuiltInDefaultValueDeathTest_IsUndefinedForReferences_Test::TestBody()) > > >>> referenced by gmock-actions_test.cc > > >>> gmock-actions_test.o:(testing::(anonymous namespace):= :BuiltInDefaultValueDeathTest_IsUndefinedForNonDefaultConstructibleType_Tes= t::TestBody()) > > >>> referenced 4 more times > >=20 > > ld.lld: error: undefined symbol: testing::Expectation::Expectation(std:= :__1::shared_ptr const&) > >=20 > >=20 > > Any thoughts on how to fix this? > > Compiling with the in tree llvm does work properly. >=20 > Did it work with exactly this git revision? I suspect an issue with the > recent google test update rather than an llvm16 issue. Note that for > every sync to github we build the tree with the llvm16 port (all be it > on amd64 by default). >=20 > It's worth noting one difference between your configuration and the > CI one: We don't set CC and friends directly. Instead we use > CROSS_TOOLCHAIN=3Dllvm16. FYI, here's a build and boot of aarch64 with llvm16: https://cirrus-ci.com/task/6710148899536896 -- Brooks From nobody Thu Aug 17 20:04:27 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRbZc1gHHz4q52c for ; Thu, 17 Aug 2023 20:04:56 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yw1-x112d.google.com (mail-yw1-x112d.google.com [IPv6:2607:f8b0:4864:20::112d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRbZb2fQSz4M9v; Thu, 17 Aug 2023 20:04:55 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x112d.google.com with SMTP id 00721157ae682-589a9fc7fc6so2166197b3.1; Thu, 17 Aug 2023 13:04:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692302694; x=1692907494; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id:sender :from:to:cc:subject:date:message-id:reply-to; bh=Vi2C5Caa7gnR6QXm4u2dAGQCI+z8aHVMMs62srK7EaE=; b=V1X5N01lhZDT6KbYKM8i3xH+KdPpMcrMOTrhtdXO96BUryTzR87oybvMaS0Ex5X+ud t0eYYuXwi31G+C8fuUlmBGvjKNhdRQvQPDHOFxORKsZP2KwRoSYtTY3Dj/GOvumnQxo9 kgGNukHSfD5b+V6U33lUwgZ/wC8doEKFoywkNzhTCm90g/yR3X9e5sLdfIYBZdE0RemK N9O75ipJdAlNmrjs3BY91svKfdPs+bfUwPyfKazy5P1p3QGEO3gx+jMor2T/YR8r4ZNI RAzKy7YVxWc+Yu1aF9Mu/mQQ7xK0QYJJjjASOxCICyFeUgwv0anhN9AX17cNnD0wHvaw rwMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692302694; x=1692907494; h=content-transfer-encoding:in-reply-to:subject:from:references:cc:to :content-language:user-agent:mime-version:date:message-id:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Vi2C5Caa7gnR6QXm4u2dAGQCI+z8aHVMMs62srK7EaE=; b=SwlnUtgsYLk6bL4C/m6q3Jnutsll9Jn3xUkU0Nuyf9VS1o+/URUvDu4erjiIAVPUPP valWXG7/NiH26Uq0RnQAtfRHe9cjejAEDR901+S0jbmGL6Uj4agAGJPcykTW1meP9L93 0H1+gQIL4yvojY4UvU1eRve7HniZ8YI/6h4a/bMzNpkUixU2Q53ITQRUx2qCdaihirTK oumADbm+SzoykDlj6DuIg8w70cwo3Y81eWz+yg3Phwhj8/pBA2CK7sJh0yHsADrv0uRn YGo0khs+nJiuj7AFwzSI9e7IHJNRE6p7WgC5GupxgJERy4krrI6m510eeXIQM2cPQg4D vXtQ== X-Gm-Message-State: AOJu0YwjUtGprO5BkUtfkQZAk8lAgWC5WUOIS1FOITdLEO2pmfUhOUET OZfOLvg2sJLQ5gBXVU23JlQnetEFwlc= X-Google-Smtp-Source: AGHT+IGfVgu+qd6R0coGPHcX8+GmuiMTIwPYGTuSS/TQIQCyLREOlft1Og3OVIF9FkjBnMh/QdofFg== X-Received: by 2002:a81:6906:0:b0:589:e586:6f09 with SMTP id e6-20020a816906000000b00589e5866f09mr420992ywc.47.1692302693742; Thu, 17 Aug 2023 13:04:53 -0700 (PDT) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id s4-20020a0de904000000b0056d31a089e9sm73011ywe.126.2023.08.17.13.04.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 17 Aug 2023 13:04:53 -0700 (PDT) Message-ID: <0fd64567-59cd-9276-cc0c-17a09611f11a@FreeBSD.org> Date: Thu, 17 Aug 2023 16:04:27 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-US To: =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> <86350hpiby.fsf@ltc.des.no> From: Alexander Motin Subject: Re: ZFS deadlock in 14 In-Reply-To: <86350hpiby.fsf@ltc.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4RRbZb2fQSz4M9v X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 17.08.2023 15:41, Dag-Erling Smørgrav wrote: > Alexander Motin writes: >> Trying to run your test (so far without reproduction) I see it >> producing a substantial amount of ZIL writes. The range of commits >> you reduced the scope to so far includes my ZIL locking refactoring, >> where I know for sure are some deadlocks. I am already waiting for 3 >> weeks now for reviews and tests for PR that should fix it: >> https://github.com/openzfs/zfs/pull/15122 . It would be good if you >> could test it, though it seems to depend on few more earlier patches >> not merged to FreeBSD yet. > > Do you have a FreeBSD branch with your patch applied? I don't have a FreeBSD branch, but these two patches apply clean and build on top of today's FreeBSD main branch: https://github.com/openzfs/zfs/pull/15107 https://github.com/openzfs/zfs/pull/15122 And if you still experience the issue, please show all stacks, or at least include ZFS sync threads. -- Alexander Motin From nobody Thu Aug 17 22:00:59 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRf8n4tcFz4qD9S for ; Thu, 17 Aug 2023 22:01:13 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRf8k6G2tz4XvL for ; Thu, 17 Aug 2023 22:01:10 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp; dmarc=none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37HM0xa2041173 for ; Fri, 18 Aug 2023 07:01:00 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Fri, 18 Aug 2023 07:00:59 +0900 From: Tomoaki AOKI To: freebsd-current@freebsd.org Subject: Status for zfs upgrade? Message-Id: <20230818070059.8325bb05c57a9b35e13d21e5@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-0.24 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_LONG(-0.74)[-0.738]; MV_CASE(0.50)[]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[sakura.ne.jp]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; TO_DN_NONE(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Spamd-Bar: / X-Rspamd-Queue-Id: 4RRf8k6G2tz4XvL Hi. Creation of stable/14 is planned at Aug.18, 2023 (Already TODAY in Japan, JST+9). Is it safe for now to `zfs upgrade `, if tunable vfs.zfs.bclone_enabled is set to 0? If not, I should wait until next stable branch, /15. (I upgrade pool only when ZFS codes are 100% match between main and latest stable. Meaning doable only when new stable branch is created.) Regards. -- Tomoaki AOKI From nobody Fri Aug 18 00:58:23 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRk5H1RgRz4qPWT for ; Fri, 18 Aug 2023 00:58:27 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRk5H0xsSz3JVr; Fri, 18 Aug 2023 00:58:27 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692320307; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y6F0wCZdmKeo9mNGAOqlye+cx5B1Adlhxe9DOu3/7Ts=; b=P3hD4PC/w6un6gwDLOmsTDFukC3spepZ+IiXoqw8/jQmkxHaAzLXOB3Yq8bywVwxw2DMGj yxnfCOUWf8/xzkfKb7vSJ5sUNCvG7vEc0J9zcfNE4iu4eOyiWiq/+vFYXrrZERo2bdDURX BDd9Xm/EVOfw5Yi1KMv8Fm79J7OQkuLnrljmIIINp+VJa26FF9ip7rU8bc0p81OveWhK8p AxIHmnvoPCgVYJbyNkDBScXkHHXp/AjtXfcovcY2bqxNuY0bcu3vFCeOcpmaK2zTJVLAeS hWQVliLrQaMuNV1E+9tMrzgBjY7rf+X/SBoOgh643o4uljM1geeAh2SMAIA4/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692320307; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y6F0wCZdmKeo9mNGAOqlye+cx5B1Adlhxe9DOu3/7Ts=; b=nqE6BV1vZgBylcFPlQ8aDWADkEn1w3z1xSbpGGFNrlZWO2kjhF4xnk4bZe6GDyJg4VlC0a BSX/OMTbq+THAfEWSMkd5nyOrC1H5vUSTzS1/kTa3FFjdxi8fGVgA3LsQuaoghRMTuCJdx RnHC535Hkc/0zLeQo89Z8k3xNQdLYfRy++81McIVefsTN7EgbVBPGvNpafhsQe24SX++Cu MAEnCCeP/EzqVyvSAWHuqc5Pf5CV93vWd107fhV7OdxPeoF1cxfBJp5OqPdbCHVug8uT9n QG8EA+wa6Lj5Z6cqNejn6ZXclb3eMqsWyafhVpjTyWd+5QLlOEjoGfdzY/nNdQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692320307; a=rsa-sha256; cv=none; b=hUiiPRPj8GabzIZwy5Di6L+Brc8TDwdYBCCyFCLbl0Y9P61/Ie28QYC6CIYWysVl0e+MJP MCk2stcj5Vt5cn3IKWAozKX49hiYvXnjMHHYdYMBKrA9tQm8NCz0K1UIv5E8FIfpGYYn6k J5U/cazTdaDkMsLM0uUOP6h6RXqJc48P1IyRQyBto6Pq1alVo4g2JOjJAv/VfkL5qwN+vJ cuPkHNk1NyPsnkCMe2IwEsa+ous+hMEt9oqjPUcD2sACzP7Mc7sXU04D5yxwcqdzat6HqW xT05EXlLw04XGlmvvBYYup5sNzTMi8RN4BCv0wgxq+nnFBZzBG4ueQ5W1rQT1Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (ti0187a400-1976.bb.online.no [85.166.95.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RRk5G698yzyCQ; Fri, 18 Aug 2023 00:58:26 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id A1742287AD; Fri, 18 Aug 2023 02:58:23 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Alexander Motin Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska Subject: Re: ZFS deadlock in 14 In-Reply-To: <0fd64567-59cd-9276-cc0c-17a09611f11a@FreeBSD.org> (Alexander Motin's message of "Thu, 17 Aug 2023 16:04:27 -0400") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> <86350hpiby.fsf@ltc.des.no> <0fd64567-59cd-9276-cc0c-17a09611f11a@FreeBSD.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Fri, 18 Aug 2023 02:58:23 +0200 Message-ID: <86sf8hnp3k.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Alexander Motin writes: > I don't have a FreeBSD branch, but these two patches apply clean and > build on top of today's FreeBSD main branch: > > https://github.com/openzfs/zfs/pull/15107 > https://github.com/openzfs/zfs/pull/15122 A kernel built from c47116e909 plus these two patches has so far built 2,285 packages without a hitch, whereas normally it would have deadlocked after well before reaching 500 packages. I'll do another run without the patches tomorrow just to be sure. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Fri Aug 18 06:05:41 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRrvs4k1Hz4qklj for ; Fri, 18 Aug 2023 06:05:45 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRrvr3nhqz4RbN for ; Fri, 18 Aug 2023 06:05:44 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.60.126] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qWscA-0007e9-6I for freebsd-current@freebsd.org; Fri, 18 Aug 2023 08:05:42 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37I65fmQ070517 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Fri, 18 Aug 2023 08:05:41 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37I65fkf070516 for freebsd-current@freebsd.org; Fri, 18 Aug 2023 08:05:41 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Fri, 18 Aug 2023 08:05:41 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: Re: www/chromium will not build on a host w/ 8 CPU and 16G mem Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org References: <2227F902-847E-4E50-B48A-B012CE51D96D.ref@yahoo.com> <2227F902-847E-4E50-B48A-B012CE51D96D@yahoo.com> <20230816163046.081be1181df386fae55065bc@dec.sakura.ne.jp> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.60.126 X-Spamd-Result: default: False [-2.80 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RRrvr3nhqz4RbN El día miércoles, agosto 16, 2023 a las 03:34:03p. m. +0200, Matthias Apitz escribió: > Thanks for all the hints I got so far. I started already the build with > MAKE_JOBS_UNSAFE=yes. It's running now for some 8 hours and has built > 32% (as the log says). So it will need some 16 hours more... After 1,5 days I stopped this again and set the following kernel values. With these the port www/chromium build fine in 14:27 hours and without any swap/page problems. #!/bin/sh # # kernel values to apply for running poudriere on my build host # Dell R210 with 8x 3.30GHz CPU and 15.8 GB memory; applied and # tested for building the port www/chromium (build time 14:27h) # # values found in: # https://lists.freebsd.org/archives/freebsd-questions/2022-February/000666.html# publishes by tech-lists in reply to # Kevin Oberman , see full thread there; # # August, 2023 sysctl vfs.read_max=128 sysctl vfs.aio.max_buf_aio=8192 sysctl vfs.aio.max_aio_queue_per_proc=65536 sysctl vfs.aio.max_aio_per_proc=8192 sysctl vfs.aio.max_aio_queue=65536 sysctl vm.pageout_oom_seq=120 sysctl vm.pfault_oom_attempts=-1 # the default values are: # # vfs.read_max: 64 # vfs.aio.max_buf_aio: 16 # vfs.aio.max_aio_queue_per_proc: 256 # vfs.aio.max_aio_per_proc: 32 # vfs.aio.max_aio_queue: 1024 # vm.pageout_oom_seq: 12 # vm.pfault_oom_attempts: 3 -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Fri Aug 18 09:28:23 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRxQ12tvVz4qydV for ; Fri, 18 Aug 2023 09:28:41 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (bastille.leidinger.net [89.238.82.207]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRxPz5G93z3XB3 for ; Fri, 18 Aug 2023 09:28:39 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=leidinger.net header.s=outgoing-alex header.b=re9EJEhR; spf=pass (mx1.freebsd.org: domain of Alexander@Leidinger.net designates 89.238.82.207 as permitted sender) smtp.mailfrom=Alexander@Leidinger.net; dmarc=pass (policy=quarantine) header.from=leidinger.net Received: from webmail2.leidinger.net (roundcube.Leidinger.net [192.168.1.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: Alexander@Leidinger.net) by outgoing.leidinger.net (Postfix) with ESMTPSA id 2D74E5B0; Fri, 18 Aug 2023 11:28:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1692350905; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=th/kz8oxOL7Pc2QTLVSD+YvIJCv6Kxbr0q+OeYA5TQ8=; b=re9EJEhR2Fa+2kXhBpxw4V2L6xz4PKk7DZOJEmUa7Pw9i4LNQbkxXCNNjdxR9MhWPbqB2c BwwyT3uwJWFHBSHRSEnSlM1cEmXsBDlH/gLczHjJlwxmyhWccCBMOfdsYZvdByrw+6qzv5 WYaIJRX3V+lCPh+RCLl17799Tp8ej3OA9jsi76B4uNEvwG6xEsZLzPsVZkfuFOwfH29lIg JmOt9Nk9NK+0A8b5Cj5BvOOsijPRVvqLl7S5po4rbyeXnxfRijuT+3vIreCBOhFALxsF2k 91CpQqJZa6vEI/eDvis8b8R/k27iIGiVH+S5oxN7gtpL/Tw0FtLze/Ij3mWyxA== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Fri, 18 Aug 2023 11:28:23 +0200 From: Alexander Leidinger To: Mateusz Guzik Cc: current@freebsd.org Subject: Re: Speed improvements in ZFS In-Reply-To: <88e837aeb5a65c1f001de2077fb7bcbd@Leidinger.net> References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> <88e837aeb5a65c1f001de2077fb7bcbd@Leidinger.net> Message-ID: <4d60bd12b482e020fd4b186a9ec1a250@Leidinger.net> X-Sender: Alexander@Leidinger.net Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-3.90 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[leidinger.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[current@freebsd.org]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; ASN(0.00)[asn:34240, ipnet:89.238.64.0/18, country:DE]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[leidinger.net:+]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RRxPz5G93z3XB3 Am 2023-08-16 18:48, schrieb Alexander Leidinger: > Am 2023-08-15 23:29, schrieb Mateusz Guzik: >> On 8/15/23, Alexander Leidinger wrote: >>> Am 2023-08-15 14:41, schrieb Mateusz Guzik: >>> >>>> With this in mind can you provide: sysctl kern.maxvnodes >>>> vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes >>>> vfs.recycles_free vfs.recycles >>> >>> After a reboot: >>> kern.maxvnodes: 10485760 >>> vfs.wantfreevnodes: 2621440 >>> vfs.freevnodes: 24696 >>> vfs.vnodes_created: 1658162 >>> vfs.numvnodes: 173937 >>> vfs.recycles_free: 0 >>> vfs.recycles: 0 > > New values after one rund of periodic: > kern.maxvnodes: 10485760 > vfs.wantfreevnodes: 2621440 > vfs.freevnodes: 356202 > vfs.vnodes_created: 427696288 > vfs.numvnodes: 532620 > vfs.recycles_free: 20213257 > vfs.recycles: 0 And after the second round which only took 7h this night: kern.maxvnodes: 10485760 vfs.wantfreevnodes: 2621440 vfs.freevnodes: 3071754 vfs.vnodes_created: 1275963316 vfs.numvnodes: 3414906 vfs.recycles_free: 58411371 vfs.recycles: 0 >>>> Meanwhile if there is tons of recycles, you can damage control by >>>> bumping kern.maxvnodes. > > What's the difference between recycles and recycles_free? Does the > above count as bumping the maxvnodes? ^^^^^ >>> Looks like there are not much free directly after the reboot. I will >>> check the values tomorrow after the periodic run again and maybe >>> increase by 10 or 100 so see if it makes a difference. >>> >>>> If this is not the problem you can use dtrace to figure it out. >>> >>> dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or >>> something else? >>> >> >> I mean checking where find is spending time instead of speculating. >> >> There is no productized way to do it so to speak, but the following >> crapper should be good enough: > [script] > > I will let it run this night. I have a 51MB text file, compressed to about 1MB. Are you interested to get it? Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF From nobody Fri Aug 18 10:36:59 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RRywz6Fzcz4qKc5 for ; Fri, 18 Aug 2023 10:37:07 +0000 (UTC) (envelope-from SRS0=NBmx=ED=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RRywz4HRDz3gcR for ; Fri, 18 Aug 2023 10:37:07 +0000 (UTC) (envelope-from SRS0=NBmx=ED=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Fri, 18 Aug 2023 12:36:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1692355020; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=B/ekUKszwcFZ02DvPD71vN4zl4amzoTOtAPWCs19qBI=; b=b7prhpUg4P2k7dZhCB92z7mx2x1TshFyjbPx6BcQ03JhtkCndwfYVICAhYc9zeFfcdP8I2 pWHQ5zNTjawQ9NQRh1HNFxb3CoIVuN1Dd4Z+XTyeDyNDSwKlcSjBfNZKJN2GMod2p6aWeB 9CNSa41KqVjmrZ4HJKU/TpmIKXDfPYmQGfjvPeBFbS3z7xzpL3sdVoqks/LgEKbDtY+Muo 8csXdHhHk7tAiDTsQkKAITrGq7hFt1KynYWnPg7Cc1XZDiOV55tn+GWSd7zQCOCwhgU0vA h9pEFfOhIga8EX9ZG3Ku7PDTMQF46mLc760UjQh2RY/zqDYu/3bFDfFALfd1wA== From: Ronald Klop To: Tomoaki AOKI Cc: freebsd-current@freebsd.org Message-ID: <792560308.218.1692355019363@localhost> In-Reply-To: <20230818070059.8325bb05c57a9b35e13d21e5@dec.sakura.ne.jp> References: <20230818070059.8325bb05c57a9b35e13d21e5@dec.sakura.ne.jp> Subject: Re: Status for zfs upgrade? List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_217_405889156.1692355019296" X-Mailer: Realworks (667.166) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: 4RRywz4HRDz3gcR X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] ------=_Part_217_405889156.1692355019296 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, This blog might be interesting to you. https://vermaden.wordpress.com/2022/03/25/zfs-compatibility/ BTW: stable/14 is delayed one week: https://www.freebsd.org/releases/14.0R/schedule/ Regards, Ronald. Van: Tomoaki AOKI Datum: vrijdag, 18 augustus 2023 00:00 Aan: freebsd-current@freebsd.org Onderwerp: Status for zfs upgrade? > > Hi. > > Creation of stable/14 is planned at Aug.18, 2023 (Already TODAY in > Japan, JST+9). > > Is it safe for now to `zfs upgrade `, if tunable > vfs.zfs.bclone_enabled is set to 0? > > If not, I should wait until next stable branch, /15. > (I upgrade pool only when ZFS codes are 100% match between main and > latest stable. Meaning doable only when new stable branch is created.) > > Regards. > > -- > Tomoaki AOKI > > > > ------=_Part_217_405889156.1692355019296 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Hi,

This blog might be interesting to you.
https://vermaden.wordpress.com/2022/03/25/zfs-compatibility/

BTW: stable/14 is delayed one week: https://www.freebsd.org/releases/14.0R/schedule/

Regards,
Ronald.
 

Van: Tomoaki AOKI <junchoon@dec.sakura.ne.jp>
Datum: vrijdag, 18 augustus 2023 00:00
Aan: freebsd-current@freebsd.org
Onderwerp: Status for zfs upgrade?

Hi.

Creation of stable/14 is planned at Aug.18, 2023 (Already TODAY in
Japan, JST+9).

Is it safe for now to `zfs upgrade <pool>`, if tunable
vfs.zfs.bclone_enabled is set to 0?

If not, I should wait until next stable branch, /15.
(I upgrade pool only when ZFS codes are 100% match between main and
latest stable. Meaning doable only when new stable branch is created.)

Regards.

-- 
Tomoaki AOKI    <junchoon@dec.sakura.ne.jp>
 

  ------=_Part_217_405889156.1692355019296-- From nobody Fri Aug 18 12:21:11 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RS1F91YNQz4qRPB for ; Fri, 18 Aug 2023 12:21:17 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RS1F84FmGz4LR9 for ; Fri, 18 Aug 2023 12:21:16 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=fMxQQ5rv; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2a00:1450:4864:20::331 as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-3fe24dd8898so8117855e9.2 for ; Fri, 18 Aug 2023 05:21:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692361273; x=1692966073; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:from:to:cc:subject:date :message-id:reply-to; bh=CZQxaKHBHRe7Xi2efoFZtGzDK1iCg4dtRxSyCHEAS98=; b=fMxQQ5rv3ZkSFnUMIs2cxmmdT++7HkSapyj9wcvcJFlthVlkiuwm08QBuGuxHZW+YQ 1OtPUsBIBtFpVWFpLVMrvcSIXCjAjJGgJRFzLKYQuIJXutgLQkTmNeqh2WF/dSvBEv2a uUS49A9WJU+ShEUoY7N2iczwbzLZG/yb7xOY+pTgkF79EQgzrcqRFvdmCYK/AZfZekcw 5j74OZgYj3bENNXLUWDE30AIXNptONm7BhIPS4Fw5GD2ZT11+SC0KkFwJJVNiOdD7DwH kUA2EPGvTdb1i1ouxfmZRPd6Pbuh4BolIWXYR/saJqBpBpPi+gHIenIk0DK0pn8f1w3L P6KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692361273; x=1692966073; h=content-transfer-encoding:subject:from:content-language:to :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=CZQxaKHBHRe7Xi2efoFZtGzDK1iCg4dtRxSyCHEAS98=; b=L+sx1Gbcp3kcj+64SQZj+v/v1/YtuOnFJbuRRG6Nhe9hB2uTudupyggvDuknPZdj2I rng3Jz45Hs0N0aB/KMjINe1A67ycwUrvM4kXFb4ZpPzi/APWckPrTyEzeKtmMPHz4G+F uvoU2XZJiBE41kT1u3UN4z1omJBvWe/iAzdQ6xXW1RqL4CSBX5A3iLwxdH0An7vt8b0D 0d81buvLEyIarst6uCOE3vcQx3UtzbyNU3mj0f3T3i50f7UU5uCPsXNZJotn0tmUC/+7 9X7voJ7NTanvtY5cKIg7cmsKmI3Y/Jv+DyzMPyYY8S6+QL4bvddlO8Ln5zvMAjQemah0 Y38w== X-Gm-Message-State: AOJu0Yx7QcOF91lbOrwTQOz6YIo0/4TFayaQzTpXykCAHaPwsR6zufBX iQc3EG2rO80mzICBwoLic6ULtDC9aCM= X-Google-Smtp-Source: AGHT+IG5LaE1bnZk3457miAArQ3gwEGWOp+ooAuLh59LInkok91Kd7Y2Z45Cycof+k5PdkQbdlye4g== X-Received: by 2002:a05:600c:204c:b0:3fe:4900:dba8 with SMTP id p12-20020a05600c204c00b003fe4900dba8mr2044489wmg.9.1692361273125; Fri, 18 Aug 2023 05:21:13 -0700 (PDT) Received: from [194.81.204.27] (oa-mowa-01-194.81.204.27.brighton.ac.uk. [194.81.204.27]) by smtp.gmail.com with ESMTPSA id v21-20020a1cf715000000b003fe29dc0ff2sm2677594wmh.21.2023.08.18.05.21.12 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 18 Aug 2023 05:21:12 -0700 (PDT) Message-ID: <3ae7e409-e30c-9b05-9f82-d286cbc8fa21@gmail.com> Date: Fri, 18 Aug 2023 13:21:11 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 To: FreeBSD CURRENT Content-Language: en-US From: Graham Perrin Subject: Boot failures with 14.0-ALPHA1 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[grahamperrin]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::331:from]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; REDIRECTOR_URL(0.00)[goo.gl]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RS1F84FmGz4LR9 I'll update to ALPHA2. In the meantime, briefly: – no visible progress beyond the four lines of EFI framebuffer information. In each case, I worked around by removing a USB flash drive that's used for L2ARC. The most recent incident was preceded by a power button shutdown, in response to a drm-510-kmod blackout (GPU lockup etc.) whilst typing in Firefox (comparable to for drm-515-kmod 5.15.25). The prior incident was also preceded by a power button shutdown (in response to a blackout at Plasma log out time). This was remarkable in that there was an unexpected restart, when I expected the computer to power off. From nobody Fri Aug 18 12:47:07 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RS1qB00m9z4qTXQ for ; Fri, 18 Aug 2023 12:47:18 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RS1q90R7Qz4PlD for ; Fri, 18 Aug 2023 12:47:16 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-88-210.area1b.commufa.jp [123.1.88.210]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 37ICl8c3047203; Fri, 18 Aug 2023 21:47:12 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Fri, 18 Aug 2023 21:47:07 +0900 From: Tomoaki AOKI To: Ronald Klop Cc: freebsd-current@freebsd.org Subject: Re: Status for zfs upgrade? Message-Id: <20230818214707.b3052896996b6db873dfa3b2@dec.sakura.ne.jp> In-Reply-To: <792560308.218.1692355019363@localhost> References: <20230818070059.8325bb05c57a9b35e13d21e5@dec.sakura.ne.jp> <792560308.218.1692355019363@localhost> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4RS1q90R7Qz4PlD X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] Hi. Thanks for the pointer! Although the data in the page are a bit outdated, but the methods to determine features described there looks helpful. (For example, the problematic block_cloning feature is not listed yet.) And yes, I've noticed the Alpha2 commit, but ATM release schedule page was not updated yet. Regards. On Fri, 18 Aug 2023 12:36:59 +0200 (CEST) Ronald Klop wrote: > Hi, > > This blog might be interesting to you. > https://vermaden.wordpress.com/2022/03/25/zfs-compatibility/ > > BTW: stable/14 is delayed one week: https://www.freebsd.org/releases/14.0R/schedule/ > > Regards, > Ronald. > > Van: Tomoaki AOKI > Datum: vrijdag, 18 augustus 2023 00:00 > Aan: freebsd-current@freebsd.org > Onderwerp: Status for zfs upgrade? > > > > Hi. > > > > Creation of stable/14 is planned at Aug.18, 2023 (Already TODAY in > > Japan, JST+9). > > > > Is it safe for now to `zfs upgrade `, if tunable > > vfs.zfs.bclone_enabled is set to 0? > > > > If not, I should wait until next stable branch, /15. > > (I upgrade pool only when ZFS codes are 100% match between main and > > latest stable. Meaning doable only when new stable branch is created.) > > > > Regards. > > > > -- > > Tomoaki AOKI > > > > > > > > > > -- Tomoaki AOKI From nobody Fri Aug 18 13:29:30 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RS2lw4bPKz4qXFB for ; Fri, 18 Aug 2023 13:29:32 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc34.google.com (mail-oo1-xc34.google.com [IPv6:2607:f8b0:4864:20::c34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RS2lw2yBJz4XPv for ; Fri, 18 Aug 2023 13:29:32 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc34.google.com with SMTP id 006d021491bc7-56ce1bd7fc4so632768eaf.2 for ; Fri, 18 Aug 2023 06:29:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692365371; x=1692970171; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=VGYKlKvPUQpnLnfgcw4jCb7cGknb3XnaaP3tbyTid/w=; b=rHBjDL6RdG1lL/VmOXB9D5W21nlt2OCJBVee1vQuNmBrp8f3m+BydKNVWBrHr1OpeZ vXzBcs17YhEQi0W7IvpVtstHKOsqx8Pt4C5ofgaaXWa62BhOBmCi9BPwlexc+aCw10bZ rEn+oNglHxf9BANZ9oaZUmax5/ioSXXTA6R0zvZy3NVmU1rPBlvbDMbdsbvKHwIJltUj Pk7uftyaH6vqQTgHhly9uDMQe+48Vsptm9C1FEAUqkUbJvv8GEerBuJeeaHpUTbALiFB wX/QRQ1XGE7Br5FZeMrkA+P8uOOzwBuGQrdO/FmoAv6F1kXbF0d5M5ZqwxtJkXxHKfnJ ktaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692365371; x=1692970171; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VGYKlKvPUQpnLnfgcw4jCb7cGknb3XnaaP3tbyTid/w=; b=SN6WRiIdkrvPv/EIU5bIQ2HLhRvkIwWypSpNP3YoyK7SNU72C+KHqHZWuOMrMVconL SxA/bBX+QNkR6zc2nZy8kdbJgTlfWzHB5im++saYYAQHymhR9w9tA50691Wt0papmA98 V5ER9sq7HTGtS8cPnOulaYfEs9KIWC+QDaeNxVOd2fBSgYMemPvU1zOkXx92MQeM0ytf KAv4PtKfXn99o7KJrdNZ9Nw9UFXi9nLMQOrbhz6i87EmnBdi03GQxmS7d4ti/1bsvHwL c2un6VEamHVmM2/W5L9EXy9zclVLPXYFny8eL9gp59vHZn8MYZFfnSts1ThjNz+uu+Tx vlsQ== X-Gm-Message-State: AOJu0Yy16iqz2quA+BGovZJFx8uIdZMs2nO13hdcB03AxxAw5mWCw24y k127zkG0M6G76aO6ZGDepNWgcTTPZTo89qO9lESQT4s2 X-Google-Smtp-Source: AGHT+IEZ5M//6ld/POlsKGaRfFFwo4AjhSREjIWckd0AsWFGu9I/YHl+qB4bsybSYF+0Voas/ZFi2+NUFzkUfrW91bI= X-Received: by 2002:a05:6870:e893:b0:1b0:408a:1d14 with SMTP id q19-20020a056870e89300b001b0408a1d14mr2996673oan.44.1692365370865; Fri, 18 Aug 2023 06:29:30 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:79d9:0:b0:4f0:1250:dd51 with HTTP; Fri, 18 Aug 2023 06:29:30 -0700 (PDT) In-Reply-To: <4d60bd12b482e020fd4b186a9ec1a250@Leidinger.net> References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> <88e837aeb5a65c1f001de2077fb7bcbd@Leidinger.net> <4d60bd12b482e020fd4b186a9ec1a250@Leidinger.net> From: Mateusz Guzik Date: Fri, 18 Aug 2023 15:29:30 +0200 Message-ID: Subject: Re: Speed improvements in ZFS To: Alexander Leidinger Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4RS2lw2yBJz4XPv X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 8/18/23, Alexander Leidinger wrote: > Am 2023-08-16 18:48, schrieb Alexander Leidinger: >> Am 2023-08-15 23:29, schrieb Mateusz Guzik: >>> On 8/15/23, Alexander Leidinger wrote: >>>> Am 2023-08-15 14:41, schrieb Mateusz Guzik: >>>> >>>>> With this in mind can you provide: sysctl kern.maxvnodes >>>>> vfs.wantfreevnodes vfs.freevnodes vfs.vnodes_created vfs.numvnodes >>>>> vfs.recycles_free vfs.recycles >>>> >>>> After a reboot: >>>> kern.maxvnodes: 10485760 >>>> vfs.wantfreevnodes: 2621440 >>>> vfs.freevnodes: 24696 >>>> vfs.vnodes_created: 1658162 >>>> vfs.numvnodes: 173937 >>>> vfs.recycles_free: 0 >>>> vfs.recycles: 0 >> >> New values after one rund of periodic: >> kern.maxvnodes: 10485760 >> vfs.wantfreevnodes: 2621440 >> vfs.freevnodes: 356202 >> vfs.vnodes_created: 427696288 >> vfs.numvnodes: 532620 >> vfs.recycles_free: 20213257 >> vfs.recycles: 0 > > And after the second round which only took 7h this night: > kern.maxvnodes: 10485760 > vfs.wantfreevnodes: 2621440 > vfs.freevnodes: 3071754 > vfs.vnodes_created: 1275963316 > vfs.numvnodes: 3414906 > vfs.recycles_free: 58411371 > vfs.recycles: 0 > so your setup has a vastly higher number of vnodes to inspect than the number of vnodes it allows to exist at the same time, which further suggests it easily may be about that msleep. >>>>> Meanwhile if there is tons of recycles, you can damage control by >>>>> bumping kern.maxvnodes. >> >> What's the difference between recycles and recycles_free? Does the >> above count as bumping the maxvnodes? > > ^^^^^ > "free" vnodes are just hanging around and can be directly whacked, the others are used but *maybe* freeable (say a directory with a bunch of vnodes already established). >>>> Looks like there are not much free directly after the reboot. I will >>>> check the values tomorrow after the periodic run again and maybe >>>> increase by 10 or 100 so see if it makes a difference. >>>> >>>>> If this is not the problem you can use dtrace to figure it out. >>>> >>>> dtrace-count on vnlru_read_freevnodes() and vnlru_free_locked()? Or >>>> something else? >>>> >>> >>> I mean checking where find is spending time instead of speculating. >>> >>> There is no productized way to do it so to speak, but the following >>> crapper should be good enough: >> [script] >> >> I will let it run this night. > > I have a 51MB text file, compressed to about 1MB. Are you interested to > get it? > Yea, put it on freefall for example. or feed it directly to flamegraph: cat file | ./stackcollapse.pl | ./flamegraph.pl > out.svg see this repo https://github.com/brendangregg/FlameGraph.git -- Mateusz Guzik From nobody Fri Aug 18 14:16:12 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RS3nq5N2tz4qZgs for ; Fri, 18 Aug 2023 14:16:15 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RS3nq3yZqz4cTm; Fri, 18 Aug 2023 14:16:15 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692368175; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yzw3E9gjfSQWJ7NhY5cXTf4BK/466gOLWpAV+hbKgfY=; b=Dcf80sp3NKiwzIyekQ3e+KlX54roctSX0U8hcakw7zYHfm+7aFrf4yyhijhnDyF+cyL4W8 Osi6o+9zi9YUvHizlOZz7S3bC8C4EoyOJkjc4x/aox2hz9PY4TpbcBVFyW/4l3KTLH6ADn Us2ZbauD4MoaHuA4Qyb8qlODSc4k0GV6biYpNo7gvdmgPlTJoHsEi6S0E48Mpes2IiwJOl 1LiPD/xZe7OimLFm2h/sRs3mxVoZAI8sCqgCST1sIF832v7vWZfiSJj+xJnrkAeZ9bWumH sd91jjnuBHr03OgOhDVU3cx++uRBcuhx5QX+/v+j2bSSkRex4IbH6g5W9lay5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692368175; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yzw3E9gjfSQWJ7NhY5cXTf4BK/466gOLWpAV+hbKgfY=; b=D3UA5tKoXBnRyo8YFjt5L9sKk7m0EnGz4xuQ+FfjxudszQHBAFcZv6cF7oWTTtWSw8TP7P 1DDoBVFgLo6J2awvxuaHfXmmOns/Da4+G3VgcGMRCuXAR0F92IiD/j/5xmoDEZlv4Hfg8P yB4tbXHoFw41ZWiKFFwn5LXgBiacb13BdN4ydviawsTu8u/IDSIUvISoPCqbWM3tfbEyON WkSgmRUmjQ1levjxQvZQbNclr0NKoSPVTB+K/VX6eSpu/SwzNCCu14aiIiKMMcaXq+fOF0 K5z3ldKHAKmQsEP5ha0MqAaB/ns73HahD36vjP3lktsnDEF7ExwNA/cDuZQwKw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692368175; a=rsa-sha256; cv=none; b=G+oZmnpNI0GtvcelOBe7UFSZRSJVG7MY4cxvmh5ggCFDdrGzuD+DYiX2Y0xTmt0Vk5AgdD /UKplHzYFyE7DjLwQ4iS6/8wGocrcigWEUw281v9S59XufZ9TrUf1jXN3/S9N+T8xNcVTu 5z30/I/4IJLxD02snYY/oWDjvAEtaMDkyLmTFSt66VoFHCDjMbihzF9rAC30J7zj5VnrwV NQ4uxq98rxTCd0CzeomzcJwD1o30/4nDBosNZ9HYHzbnPg2mGQbSO8qsUTyh83zYvt5hWu jvGeEp6YJu/F5CSS4xmGr0ZvM6B2YfvSBPHYl7ejGFqDKFYHuueyaolYm4sk2Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (ti0187a400-1976.bb.online.no [85.166.95.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RS3nq24Zdz1D6h; Fri, 18 Aug 2023 14:16:15 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id 6B1282825E; Fri, 18 Aug 2023 16:16:12 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Alexander Motin Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska Subject: Re: ZFS deadlock in 14 In-Reply-To: <86sf8hnp3k.fsf@ltc.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Fri, 18 Aug 2023 02:58:23 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> <86350hpiby.fsf@ltc.des.no> <0fd64567-59cd-9276-cc0c-17a09611f11a@FreeBSD.org> <86sf8hnp3k.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Fri, 18 Aug 2023 16:16:12 +0200 Message-ID: <86fs4go2qb.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dag-Erling Sm=C3=B8rgrav writes: > A kernel built from c47116e909 plus these two patches has so far built > 2,285 packages without a hitch, whereas normally it would have > deadlocked after well before reaching 500 packages. I'll do another run > without the patches tomorrow just to be sure. Plot twist: c47116e909 _without_ the patches also appears to be working fine. The last kernel I know for sure deadlocks is b36f469a15, so I'm going to test cd25b0f740 and 28d2e3b5de. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Fri Aug 18 16:17:42 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RS6VF0kztz4qkYZ for ; Fri, 18 Aug 2023 16:17:57 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RS6VC6nWBz3N5M for ; Fri, 18 Aug 2023 16:17:55 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [188.174.60.126] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qX2AW-0000MB-Hm for freebsd-current@freebsd.org; Fri, 18 Aug 2023 18:17:48 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37IGHlIh003171 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Fri, 18 Aug 2023 18:17:47 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37IGHgoI003170 for freebsd-current@freebsd.org; Fri, 18 Aug 2023 18:17:42 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Fri, 18 Aug 2023 18:17:42 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: make-memstick.sh creates in 14.0-CURRENT run-away processes Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.60.126 X-Spamd-Result: default: False [-2.77 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-0.997]; NEURAL_HAM_MEDIUM(-0.99)[-0.992]; NEURAL_HAM_LONG(-0.98)[-0.982]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RS6VC6nWBz3N5M I was used to use in 13.0-CURRENT the script "make-memstick.sh" to create memstick immages to install the system on smaller devices where the OS can't build from the sources, and it always worked fine for many years. Now I'm ready to do so with my fresh compiled system (sources from git August, 5: $ uname -a FreeBSD jet 14.0-CURRENT FreeBSD 14.0-CURRENT amd64 1400094 #0 main-n264568-1d7ffb373c9d: Sat Aug 5 17:22:47 CEST 2023 guru@jet:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 but the image is not produces and some processes create temp files of 800++ GByte. Here are the details: root@jet:/usr/src/release/amd64 # ./make-memstick.sh /home/guru/140.root ~guru/memstick.img Calculated size of `/home/guru/memstick.img.part': 23795073024 bytes, 263113 inodes Extent size set to 32768 /home/guru/memstick.img.part: 22692.8MB (46474752 sectors) block size 32768, fragment size 4096 using 27 cylinder groups of 869.44MB, 27822 blks, 10240 inodes. super-block backups (for fsck -b #) at: 192, 1780800, 3561408, 5342016, 7122624, 8903232, 10683840, 12464448, 14245056, 16025664, 17806272, 19586880, 21367488, 23148096, 24928704, 26709312, 28489920, 30270528, 32051136, 33831744, 35612352, 37392960, 39173568, 40954176, 42734784, 44515392, 46296000 Populating `/home/guru/memstick.img.part' Image `/home/guru/memstick.img.part' complete Creating `/tmp/efiboot.iFachZ' /tmp/efiboot.iFachZ: 65528 sectors in 65528 FAT32 clusters (512 bytes/cluster) BytesPerSec=512 SecPerClust=1 ResSectors=32 FATs=2 Media=0xf0 SecPerTrack=63 Heads=255 HiddenSecs=0 HugeSectors=66584 FATsecs=512 RootCluster=2 FSInfo=1 Backup=2 Populating `/tmp/efiboot.iFachZ' Image `/tmp/efiboot.iFachZ' complete It says 'complete' but never ends growing the file /tmp/mkimg-oGNnFb: root@jet:/usr/home/guru # ls -ltrah /tmp | tail -6 drwx------ 2 guru wheel 512B Aug 18 15:43 tmux-1001 -rw------- 1 root wheel 33M Aug 18 17:18 efiboot.iFachZ -rw------- 1 root wheel 0B Aug 18 17:18 mkimg-4eMWKW drwxrwxrwt 21 root wheel 1.0K Aug 18 17:18 . drwxr-xr-x 22 root wheel 1.0K Aug 18 17:43 .. -rw------- 1 root wheel 850G Aug 18 17:53 mkimg-oGNnFb root@jet:/usr/home/guru # ls -ltrh mem* -rw-r--r-- 1 root wheel 22G Aug 18 17:18 memstick.img.part -rw-r--r-- 1 root wheel 0B Aug 18 17:18 memstick.img Only a hard reset and reboot helps. What does it mean? matthias -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Fri Aug 18 18:05:35 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RS8tp2K7Kz4qsYN for ; Fri, 18 Aug 2023 18:05:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RS8tn1Gbnz3cn6 for ; Fri, 18 Aug 2023 18:05:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=cole0q5x; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692381950; bh=f5sjZZIV5u/kVPwPpSdI+5pKEi4V0DfQlJC9ygLqPAs=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=cole0q5x40kU/knvHcmku50ZVV8uUsClnMTRNkKrMNPu+FYWxbwd6ApIWJu1F57aUtqY9ShxY22jSDYcoqxIcauN+Wbt9v9e3fJ8CC4Uyd9+aAnLpG3p+JslcM110nsI/kX6IE2BwRQQp3s6EIkFUmjUUzHn/197OPmjQ1LaBwCD22JZq4zb7fWQ7DXCdWNITQ03x8EIf7DGIhsuVpwbrqhKdSGTRCDIV9vrnLylf1GdPbIq8IVN7BrOflqSCwoQS2EH1viRsdwChMeBrxJ1Y/ZkdLGD63r35Y6MCREXL0T+Gr5oYsy9jHwHwBYYvdnlZx6iwzDrz/y2iH43wz47/w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692381950; bh=FBFlH5woSjM3WFqTDq95bdNCevmnUU9KlMMo2lyX6bs=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=ndfV6l5NjSnHVGciu9a0CEB2nBhj/HzibvnsUeZfZJwt14t9Wsinn/lMqghjuDiJIBUOFbJeW5JBc9FPekbvhMcLoYOpEf9Phl4xjkP8WFr/uv5KE2B0jl8L4MkEp4dH7kDbpcR35EtLK/8adP6CtTeiEbiQgqtg2s7ooRW5l8uKHH98S/eYKwxSiNcNaxCvKdvr0V3K9RotIL5HSz1v6Hr6PToqa4PqPyA3C/HPetr4F4NeHK0drLPzanGUXQ6CL4pnC8HVafc3XcQECLDplCZxt70jeLA2y2k8ubWZ6ZHYgsWQFMMFfJlP1kmTjaXiJaV+Q5sauQy2ft7VKIHnMQ== X-YMail-OSG: IKaasNUVM1nQlw72g.qTpvPijxQttEms.ObSRH9I4HieRMLe8tFzVGBGb03UwKp WvvMsJpX6YrnTG83i70qJA1Bve02.V0BRFkLUsZHOklg.zocg0Um6MYziLXCLsoY21M9cCDw49Qs qEtmiN_tCtw83VQpV54G4dhYKZqQrFuLDf3HjZJIIv8rq.ke.sOnbp5mD71Q8TtoA6Fe34jnJwj5 mPuFROHbyVBaktAdLRvOT9zw4an9B2JAfaPxaB0cHlDna5hYcp6DxqUiEhsBieumBFOyfylydpfo s_e27ZQrBk.UNJoxaWZZrhOIxnsAhCfg5xb3cudWgzftQm3MRv_b8BDxKJG3gy1HNwah190t9Ysw lEPC_IBdLEX9qzJFJlkp0bveHWWhOaJZ8.HLRJmHeXvhHL9wl_Wsn0NMeEf4QXRMifk8_7aQh71r U4PkPRVrL4kKvIyn0caqycrfqb8Vhk0JYW206WbAIxWK2gVyIJFqXX1RD_rRprI7zaYqX4Iectvp vZo59SHJY6Y3LP8p6ncbp2R1o.SpKJR_x91rOTcwoWINj_ZwLW8iE7qnXUexNlm1TZLYCg3D8CsN isCQgWd2322Ia1x.GaIoSAhvI_i7vJDUavDM1ngQIa88_emPDn2a0_9wPNtMyDiyPtEkUi8FYyhj 12nv5v1WYIBVN0l_k0eQ36DWx6jN99TS7j229VpfwSNLqfmwbod_53j_6_6Vo.oXddencnx17sN5 CQ4m3msrXjjGQgiL5vY_ptsMqLrRuD6HiHpRcBp7S2cwUdpU9VxHjbnV9CwrENPhhS11hhKAyz4l yQcGMerSFbQGifk13ApnP3ZIbGYcRWV7eiP12A6SuIKsh1Pgv2bk3DAuQGuL244g1laHE8AhdkBr CyHaQtgtaX.4Y37LMDV7yuexWqaAAt3nk3soor4aqpZmIcyBfkyOn2hOZpYI4WdHHbvwb0oYGnfY oUSvSxd4v9Sp1WGp958VzJbDtMTVo3_4PekiMBRTtO_D4ayrruQEGCZ7xCndI4fBoAzvae4OK9Oi lwT5VJyjgu.9mZh5aKRmS_Kn5n1AwIjPhx3Yiw597eVkO5UDHQfgXX.joQemGYBGfzOO3sagXBqp NbiDzIu8ETS3BAMsHCUx_xCw4d5pfIg79d7MK.tsXl7NLdQlRSRSWF1iWxBZN5dRasWXA1F6vna9 prgTya_1zf0TkgnBzi4DvHt9tNon8Z1p3NdJ1ZmmiOg0taebMOroj2ea3G6UuGchnqymlY741oHr RUiSy2jX_L.J2Uan1qyFLdIO3Ph6Xtl4weFxx3fQnQbnWDtP3N9COfv6kD3p8VyNkp2vVB.X75qT B3vazHrcYnnAEdKR3ON8mN3dS727g5Xl1nkl4zb06tRULXsSy9iuRHD2AA_nwRW3j8XGWlPb3wQd 0tKuBiyAAfr9sVeCKa1RDPlxOGYat854bOgEWARIT50xcV3TZjq5WDTgRAMMxm4rjnuX4eDF4aZ. epkq5_RbkOB735Yif9lM9UQNLSjoVyyZsN8_eBdK1z1rthHOC9TMQ7MJATaPWJAvwXkNjWhCk7oC h4eUS2mM5B84ynQVvwjXqyLYK51TJC57Pkx3frEh1sU.hDe1.sYCuLjVmcoaMCw.9DPZ_cZdaet1 g3l3A2cRafJCZbiEvtqgVwAAURsyu1ebwIzF9cydZ9nfdm_ENBbye2JuXG0OagDXTu3RuFaO9d0s s6K8BtdSTkJAkTrdrkYh_5iEyVyoScIHllvb1iN3rZuYoC9oJw_YecIz0NyrStvwC3IAXcE8_xyO eqcCqFUmdMaj8.XtUORY9LZY6uCySZ97k9aP_0PDTDCu3XbmHLaLK38P1PreCV6xCgY821Boy2di LyZkKIACt8.JPrxMv6hv2PFBFrLZHeqgb_sIjSD7rau0Y1f_CFmycMqmjWG98fFhfNATcOpWRlp1 FLrM2aftdvMETWxHRxAcnNtUIocyZeCBlb24BRjGV3sAUGIGA8KDTKDWPk7dkNgOAFleT_BNcVWP 0x2ilS5.F6G8PRS6ia45oJLaBM1T0XxeTgfegiThKr4HUKqIUj6rbisRMc0AnjNDTS1iV8DjsEfg 5PadOWnpumUNuKJaX1iN68acz605xvbWc3pshAo3K0troeRWBLtZKMKcyraELtlvLcl.Oh0XYbUA RFXHizcIgBuESiOCNCiC3ErFcosv65nmpFrtQWpB.k0ROKm3PThj_R1dDa._GOrAkaBy11dUYznb hkSgKYT0CAdL49iiPVuHptr5srGQdhlwAmewLRg3vjPp6ijrVuk66ILOZW.hEsw-- X-Sonic-MF: X-Sonic-ID: 49c4b01c-4538-4c5e-99ce-a287beb4aa0a Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Fri, 18 Aug 2023 18:05:50 +0000 Received: by hermes--production-ne1-7b767b77cc-6mpms (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 6c61045bc5ca1d40a77fd5c903efe8a9; Fri, 18 Aug 2023 18:05:47 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14 Date: Fri, 18 Aug 2023 11:05:35 -0700 References: <5B44567B-BD35-4E37-A016-0EA71D0EB9F1@yahoo.com> To: Current FreeBSD In-Reply-To: <5B44567B-BD35-4E37-A016-0EA71D0EB9F1@yahoo.com> Message-Id: <38A2B0B7-4B9A-4919-A270-75242070B10E@yahoo.com> X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.83:from]; RCPT_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.83:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RS8tn1Gbnz3cn6 [I had sent to the wrong list. Just fixing that here.] On Aug 18, 2023, at 09:26, Mark Millard wrote: > Dag-Erling_Sm=C3=B8rgrav wrote on > Date: Fri, 18 Aug 2023 14:16:12 UTC : >=20 >> Dag-Erling Sm=C3=B8rgrav writes: >>> A kernel built from c47116e909 plus these two patches has so far = built >>> 2,285 packages without a hitch, whereas normally it would have >>> deadlocked after well before reaching 500 packages. I'll do another = run >>> without the patches tomorrow just to be sure. >>=20 >> Plot twist: c47116e909 _without_ the patches also appears to be = working >> fine. The last kernel I know for sure deadlocks is b36f469a15, so I'm >> going to test cd25b0f740 and 28d2e3b5de. >=20 > I'm confused: >=20 > Tue, 18 Jul 2023 > . . . > =E2=80=A2 git: b36f469a15ec - main - zfs: set autotrim default to = 'off' Yuri Pankov >=20 > But you had written: >=20 > Dag-Erling_Sm=C3=B8rgrav > Date: Tue, 15 Aug 2023 17:37:45 UTC=20 > The attached script successfully deadlocks 9228ac3a69c4 >=20 > where: >=20 > Thu, 10 Aug 2023 > . . . > =E2=80=A2 git: 9228ac3a69c4 - main - ixgbe: Add support for 82599 = LS Kevin Bowling >=20 > (You had previously reported a poudriere deadlock there as well.) >=20 > Later on 2023-Aug-10 there is (about 11 or so commits later): >=20 > =E2=80=A2 git: cd25b0f740f8 - main - zfs: cherry-pick fix from = openzfs Martin Matuska=20 > =E2=80=A2 git: 28d2e3b5dedf - main - zfs: cherry-pick fix from = openzfs Martin Matuska >=20 >=20 > I'll note there was also this: >=20 > Dag-Erling_Sm=C3=B8rgrav > Date: Sat, 12 Aug 2023 14:11:10 UTC=20 > . . . > Trying to narrow this range down, I did not get a deadlock with > 4e8d558c9d1c (10 June) but I did with b7198dcfc039 (16 June), albeit > after building ~1800 packages. > . . . >=20 > So the count seems to sometimes go well above 500 or so. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Aug 18 18:46:52 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RS9pB47Zqz4qvs5 for ; Fri, 18 Aug 2023 18:46:58 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RS9p93sMbz4FXX for ; Fri, 18 Aug 2023 18:46:57 +0000 (UTC) (envelope-from guru@unixarea.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of guru@unixarea.de designates 178.254.4.101 as permitted sender) smtp.mailfrom=guru@unixarea.de; dmarc=none Received: from [80.187.83.94] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qX4Up-0007a6-0t for freebsd-current@freebsd.org; Fri, 18 Aug 2023 20:46:55 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.16.1/8.14.9) with ESMTPS id 37IIkqVA013647 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Fri, 18 Aug 2023 20:46:52 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.16.1/8.14.9/Submit) id 37IIkqAH013646 for freebsd-current@freebsd.org; Fri, 18 Aug 2023 20:46:52 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Fri, 18 Aug 2023 20:46:52 +0200 From: Matthias Apitz To: freebsd-current@freebsd.org Subject: Re: make-memstick.sh creates in 14.0-CURRENT run-away processes Message-ID: Reply-To: Matthias Apitz Mail-Followup-To: freebsd-current@freebsd.org References: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Operating-System: FreeBSD 13.0-CURRENT r368166 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 80.187.83.94 X-Spamd-Result: default: False [-2.78 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-0.996]; NEURAL_HAM_MEDIUM(-0.99)[-0.995]; NEURAL_HAM_LONG(-0.99)[-0.993]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:178.254.4.101]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[unixarea.de]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; HAS_XOIP(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; REPLYTO_EQ_FROM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_XAW(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4RS9p93sMbz4FXX I filed this PR: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273211 about the issue. El día viernes, agosto 18, 2023 a las 06:17:42p. m. +0200, Matthias Apitz escribió: > > I was used to use in 13.0-CURRENT the script "make-memstick.sh" to > create memstick immages to install the system on smaller devices where > the OS can't build from the sources, and it always worked fine for many > years. Now I'm ready to do so with my fresh compiled system (sources > from git August, 5: > > $ uname -a > FreeBSD jet 14.0-CURRENT FreeBSD 14.0-CURRENT amd64 1400094 #0 main-n264568-1d7ffb373c9d: Sat Aug 5 17:22:47 CEST 2023 guru@jet:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 > > but the image is not produces and some processes create temp > files of 800++ GByte. Here are the details: > > root@jet:/usr/src/release/amd64 # ./make-memstick.sh /home/guru/140.root ~guru/memstick.img > Calculated size of `/home/guru/memstick.img.part': 23795073024 bytes, 263113 inodes > Extent size set to 32768 > /home/guru/memstick.img.part: 22692.8MB (46474752 sectors) block size 32768, fragment size 4096 > using 27 cylinder groups of 869.44MB, 27822 blks, 10240 inodes. > super-block backups (for fsck -b #) at: > 192, 1780800, 3561408, 5342016, 7122624, 8903232, 10683840, > 12464448, 14245056, 16025664, 17806272, 19586880, 21367488, 23148096, > 24928704, 26709312, 28489920, 30270528, 32051136, 33831744, 35612352, > 37392960, 39173568, 40954176, 42734784, 44515392, 46296000 > Populating `/home/guru/memstick.img.part' > Image `/home/guru/memstick.img.part' complete > Creating `/tmp/efiboot.iFachZ' > /tmp/efiboot.iFachZ: 65528 sectors in 65528 FAT32 clusters (512 bytes/cluster) > BytesPerSec=512 SecPerClust=1 ResSectors=32 FATs=2 Media=0xf0 SecPerTrack=63 Heads=255 HiddenSecs=0 HugeSectors=66584 FATsecs=512 RootCluster=2 FSInfo=1 Backup=2 > Populating `/tmp/efiboot.iFachZ' > Image `/tmp/efiboot.iFachZ' complete > > It says 'complete' but never ends growing the file /tmp/mkimg-oGNnFb: > > root@jet:/usr/home/guru # ls -ltrah /tmp | tail -6 > drwx------ 2 guru wheel 512B Aug 18 15:43 tmux-1001 > -rw------- 1 root wheel 33M Aug 18 17:18 efiboot.iFachZ > -rw------- 1 root wheel 0B Aug 18 17:18 mkimg-4eMWKW > drwxrwxrwt 21 root wheel 1.0K Aug 18 17:18 . > drwxr-xr-x 22 root wheel 1.0K Aug 18 17:43 .. > -rw------- 1 root wheel 850G Aug 18 17:53 mkimg-oGNnFb > > root@jet:/usr/home/guru # ls -ltrh mem* > -rw-r--r-- 1 root wheel 22G Aug 18 17:18 memstick.img.part > -rw-r--r-- 1 root wheel 0B Aug 18 17:18 memstick.img > > Only a hard reset and reboot helps. > > What does it mean? > > matthias > > > -- > Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 > Public GnuPG key: http://www.unixarea.de/key.pub > -- Matthias Apitz, ✉ guru@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub From nobody Fri Aug 18 22:34:20 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSGrc2MZgz4rC4x for ; Fri, 18 Aug 2023 22:34:24 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSGrc20SSz4hJ0; Fri, 18 Aug 2023 22:34:24 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692398064; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CrVK2iwVPpZ4GBXa1QHJhdCtEP3wbALJZObEtFzKrH8=; b=AQhKtK5ZM9CsMJ6bMKNGpM24lOmToCjHTgwuQxzK0nG9hQLLvBi3P1OaGNGqxGRR485nUI MY06gqiF02th0Z/ooRrvOPXwOUFmDzpyYTwjvq2lZgw7sWITxht+C+bdZvVFP0sCbQ7D9X PYOQcJpkz53aJHl5hih94XzOAR4NxpJPxAA2Eb9pB9kIVB0gLRApVd6B+WJIiUgaDEUgjd 4QIVIRelY88HazvYQlbDI/ZlCMBeVDHzu/svkUy9feZFRihSjf0B2Z9CJ8vA+29iuKa+ZB H6z7ZHcAcSJGHCaggVbM10YCGGAlQlkffrccjaY0lGzGWf5ED/21mkaUaFB1aQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692398064; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CrVK2iwVPpZ4GBXa1QHJhdCtEP3wbALJZObEtFzKrH8=; b=sQC8AWw33tOgMPxo8xIrS0u8Qfbfh+BHfqUtMEj8p1s0A/vR5r74HlsRzBsMune/SndP3d EwxMUKsnKjsub/rAIK2k6uydyj1D0ppsEjN+jpfCO9qzBOcMogmSH7nOBvWI9MckszT2wC K6sSx5VjCe3fDs4XhRB4fDpa4Sq73/MFjZBIH6DkIQbcQ5/Ua4J5wzpNClwdBYGivxYCo7 pfxvXe6vq/1LjDBu2nEXpVlDXMD8cvnNmxtyxV9Tyn34A8CcXjMhfJtRaSXQKA6l7XtWw3 QC1A8c7wg0Vx5tubV2V0yI551fJY4ZdqzUwKSgDbVdqDCn/ZrY6+FcMUO64Tqg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692398064; a=rsa-sha256; cv=none; b=nKz37ZCYI4E6NEVnzlih3SH6vtPICmVepazT6hKeJmh1bpV7o0/OBBn8doAyGs+JQnb6AN Y2yfEkilXmk4xbOev0ev78W2+oTIOKgiMjo6877wkK4v4UXhpixyTgBtNMjyoc6o2i1OKY JOZwzfpwmwoho6dePEY7myCwLpVtmRhQ2hCHfYBvr7L60oyNOW+1cOwaOfhkFFp0JdFPm4 6MnNn7gEUgLsw2SU79Jni0PTPGfjSDibpwuh3BNut+y8Uah7EyNtbju8sqngKkWxuekRbR yiR5XGZZpPRLzaQNjnXXzAt4j5Q9+7i/hh0DjhDWbUKxDtANQT+dY9oSKVDkZg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (unknown [IPv6:2001:4647:d671:0:36e8:94ff:feca:9834]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RSGrb4Dy6z1MmS; Fri, 18 Aug 2023 22:34:23 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id CA27D28C83; Sat, 19 Aug 2023 00:34:20 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Alexander Motin Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska Subject: Re: ZFS deadlock in 14 In-Reply-To: <86fs4go2qb.fsf@ltc.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Fri, 18 Aug 2023 16:16:12 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> <86350hpiby.fsf@ltc.des.no> <0fd64567-59cd-9276-cc0c-17a09611f11a@FreeBSD.org> <86sf8hnp3k.fsf@ltc.des.no> <86fs4go2qb.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Sat, 19 Aug 2023 00:34:20 +0200 Message-ID: <868ra8nfo3.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dag-Erling Sm=C3=B8rgrav writes: > Plot twist: c47116e909 _without_ the patches also appears to be working > fine. The last kernel I know for sure deadlocks is b36f469a15, so I'm > going to test cd25b0f740 and 28d2e3b5de. c47116e909 with cd25b0f740 and 28d2e3b5de reverted deadlocks, see attached ddb.txt. I'm going to see if reverting only 28d2e3b5de but not cd25b0f740 changes anything. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org --=-=-= Content-Type: application/octet-stream Content-Disposition: attachment; filename=ddb.txt.gz Content-Transfer-Encoding: base64 H4sICDXx32QAA2RkYi50eHQA7L1dj+Q4siX4Hr/CMU/Vt6e6RfFDVAG3Zxf3AosdzAKDubPYh8Ig IIlUVkxGRkRHeFZl1a9fUjK564NydzOjVzcaXY32jPBwOy6Rx0yk2SHp2h+KHz679k/+5ejf//TW vDx1fzkc3r++HJ5fu89PL/3rg2t/ED9Mv/3l8PHT6y/DHz8e/Lfu+evH08/+8PHs/dvhy9ej/3Z4 e3/t/MfH8JnDd/Pf/nB4P/zroTh8V3zr43++UKaSlTTW/mH4gHeH/+Pw57fXr+79yb/7P//v5un5 489CNV+cUX/++vH+54/37s8fv378+bN/fxleHrto+Kfuh9JUDx8/Ne8B4+PboXl+jt98+A5+WH91 +M8KU5RdUWT4al2tW2lspvDlY0v9d2gFY5QUh+/eP359CZd0/OndN+4wb42ukMKHyxTSVlL+YdbE EejLp/fDb/3H4bvwsr4jW2jvSlWWknZHP/cfjx9f2/dwP8JoPbXl9a8Vum+EbHr613ZN95MP36uL 8L2zpirr/aaSQnW9V7GpCmGtwDWVr6zrVVX94zSV2m+qypiqr8qBVbIWGtdUtaqELut/IFbJ/aYq vBTN6IBVUegK2VSi9r3+R2JVebGp6kKb0QF1bbFN1XW1+QdilbgQq5QtSjc6oNJCo2NV2ZtOt/84 TVUdvvv4KRnTG9/UxRCoTGGQlGpCQwnx+z/+ClvVsmN0z247yZ12Cq4Xnl3js68UKvDp9qstveoc p5GeX18/f32LzVRUana50prDd86l6O9ceFAP3VrJQtdI+hvf6OAEeboV881CaqMajuPttlWldl1A uaKzMKwxBaqtRFFIpf4GUfVeLiCVPXz35bN7ek97QdmN0SI0lcA9q0UhfSVU8fvTShRKhGdAhm+W 68YKcfXT0zHRVFYo2wwPoFIbWxtsUylV+lweeDOrSl/orsswmtq0k9lrp7bxVg3eF9upwrZTGFo6 w3EDUjvJorS9zuD0m3bSh++eX1JByrrajHMvVRUWOaHwxlldcQYWlz3v+OUtfvfwz/bbXVfruuO0 1hSrlDFi3lwyfM2z+9Pzc/IZ2FaqLYYhYKWMxDJLWNcY+bdpshDdRRkeLQxH3Guy0u6NcFzhy3Ee ZmpbIhkWXgonc40EsQzzUuta5Bg4r5qr1HIvcIUvNW54FsbApbHDBhHHzr/7yLl0bRd8kR8vV4Gr lGp/1BAGWH3ddOOooZYldi7WtKbMFeLRjqicalkPmD1mFVWMXd3rS//0KRW9qq5rxugVBgNaYQN+ 402ZKy2C+eYQA0pf38EVw0Tm8F33xz8mCdaKehzBV0LUOFcMBNN1b3NN9tEEM03RtJznzG57lTuR Pqa81TiXDePiCkutSjaqvd/k8HJzhUBfmIrTWXvNJetLEUwrOUawKkwr0RFMGtPczxmvEazzYfab fyghZBjUd8nkmwmTRNkMSaXgkOjBaoxdf4vsQxtGyR1rELPXVDGruz9QDc/lfmSXVELjBqqhuXrl zf1m1VfYpV2YmtwhfAUvD/54fPc+ObTXVTkSTBVS4rKW4Zleh1HI32isGvyxL9viDg/IQgeS/W// zSddsq16yF9WYVSBi/lxmq11/7unI0z4XmVz+OMqyVXE4ereWKKx3Zg5FXVp6k2i90rvls51rJn2 TgIzzG/C+Oft86fvP47N8SnZx8438JSStRDYCa+WpnZ/i7CrvdO1yv9M19bsDhmjP9ixPiRsrS26 m00cRuXPU+sqpoDS11wWbVdONYhAUawP943xv3elNAzYhG7LHA+IpQ/roSHSDVW4oi8g2AlbYB8P uggtlStZ9nfQUHFkuztx6twY7KQVFln9C+Pa2uRIJuCeCtr3jWPViHYbyu43lOrdVHm3FilS8JXr 2t/98XlPRlW7ruelriFtIazAFR5DjGpdZ39vRt2zoXaDeRg1WzM+rFV4XKOnlK7y/1CM2n1SB0ZJ DUoOVaILaaF1XfGPxKhyP0b5rhmDuVLaItM6tdGt9f84DaXsboyKBX+odMR8tPq7GeKXsfZ+cYgf HthtDQ9sUyIL8EVbm7JkNTa5AG8aUbOqaTtDfFHsDm6iWGFKc6o6Fmj+PrpZ1bXZnZbY0lhINIpC Vn8311zZ3Rlz0bpewrRECLmVRv3NrrkQu5Kftushf1QZgR73ql5lE2dgvjkmRITM70bK2N2BSpQy m3Hoq0pd/v1QUuvwPV+az/6Xn0K0/NipodUQ6KVCJlbDYNQr0+RSduFmoL1UrJTuTjeLKBhMeoRy va1ODxaJf7Co3vxtcke+lzZ/U4VZzH7uSKjwDIYRqUVnn3VZ183fLPushInJkfztVew+ICK37Fie Vboo8Q+IO+XaZFnvjhFjrq2dHmq1wObLdSEdqwjO0lsVTWmyaEOXfVyqYjflFqazsUQ69PGohvn7 6GNR7/txmDBJNw62hCgL8/dzzftpA2Prxo+PtPCEqP9unsbhN31l7hLG49YUIB7WBpfxiOUJma+I fvP8tDCd6lmK2PT8tCijdPhae1V2Wm6na2wICsyv2vupWi42WW9YhaRk+NF1Ve2Ga+VNGJGPc73C EtziPq6sa1Fc62YpVNuPbmFkYdD5LdmK+m8w8grdLEpfZ1f1amv1hYitnAAlYVXbv5eRhA5TJL03 1azrvoPljVKgvViZOMj93ReiSV01InvU08qavSl5nGxBtiaMt5AC99BOnajvKBK9oKitu7rosk/J dSHEhbKddxXoq6Swfy+DrTIutbz+UKvBg6XFLrYqvG6r0uZKYKIaTbaN8i77gitbF7E8+/br8afX F/mnOinXtEbD0Fpo5A4FsQBSSPl7R5DwdLDhWnOvJKpUefju5fiWUtHF4ZIYmykMl5C17BBAwvRF 329WfiHWlr6qXXZ9pi6DK7bPTff5+enjmGwwV1g3jccFen1MiFAmmwwYs0RaeMWKAL9+dM3z80fk llb1qcFUlM+99c+vnxKNVXaucPUkABBy8Rj/+Hb4eH18735+DD99d/oxdfG1abquo13816e37vEj 2h1jZxfSnK49fEn8iH9OXHmrKtOAXxTaLqZd/9xDJrmHzAj888ur+3j4b+OXxF/8R/x4avOeweqt e/v6EP7/5A7w378eSvvgfn1pvjx1w5/jvUxtKFrr2ibMWR66r+/Qa4PNtut+OLwF0OJwDK9jJx66 96fji/84HsThP7lg+vz53X/8pwj11rWnr19B2a546N9mX/evh5fXF//w5J796QoWVrLWXWD9D9NX FzJ8X/z4D4dwO6X9Tw8f/rk/HNbfGPupbGI3Dbf39qV5S3xC1EaXrng4fny87WNIqx7eP96K7Sei XzZeRi3cw+fuXSYxpv8evl79xMf6E8XDpw9ZvqWMhktThXp4dsdDEnb8hFLx9i59QqrYRD+/+OPp Ew8fb0/jdlAfh5/8s/thj3jt8eF/vjfd08unLUWOqSjwEDAeB4zv/nBojofTr38svsnyz/1788Wv mle4snj4efjG0Wb8ORiECWvawuviYWYwfV7tfbwtHk4kHk1Ovw7X1ezY9YHPr++fH/23p+Nod/o1 2Nmd++klmB3DX99en59e/Mz4/GaA8PsI33///SF89i38ret8Z3Wl/vPh/elt6GHb2z5ESFXWynVW iPCXj/EvZeN84VXXVNq3pnDhL+34F+Nk0btOdMbo4HjmEL5hr9/fPh4OQ48f3sbXT++RpUPsiWNv H3755Yv/+BT/7X5qXg7L/7ov7sEYpYu46YWyh/Hn+F95OPw70DU+heN/86HKuJNFjHgPo+XhYIvi 9Bo+fDj8B3zJ29Obf3eHxRitDtbxk93768vDuDXX4VBV5el1QPj3w/kKjr45X4HurY/DO3U4DDu/ PIxbVrEhFB9C8iFKNoTgt4Woho1QNGwbM0F8XIcYOREsUZwQNmr+rD1zQsjhu0vYkOUqK6c9Q05X UCKvoOydml9B3Fclvkp4na7ghmZ07uFsWS7t/wPu4Jfm6Ti7g6FQXQW/UNMdREukX1lXd1Uxu4Po VyHwnF4D0Bnhu/Fp78NId/PfT8cwv3oQUlgttn89QXz+6zheOHdDJWoVF61PEKU2tVAXIMLQwXfH GUQvW+GM7JYQGgERvr5z1rZniNgWBbMtijKMOS9cRXxQx0HQuS288pUt7PJGSuSN9M61agkhcRAu zGqrRVtUNbMtlKztpR7ZtkUlhKjtiheXmjNxI97Yvlu1xSV2biHCxDFMmdSiLdSwL1AJuwPF/zCR TlZIL40DHV/XszgTn7lhNimnV7iCP56v4DBvylOucbqC8PyNCWZ9kKGFw91oLVW0/69X72BY6fow 7mCjK6tqeF1dwiVCfHo6BjrEHZMudMRXCHbzaN02xsT2igDj5jgJgFuCbQQYN5dRVoTp+/iKvoVK FKkr2L8FXUvXOzu7hYp7C3rY/aWYXm/vyOeBSMGmqqyul/b/AfbbB47QWg06QyCSHL45jqTCbKpE ff+wqvVhtDbKxg6YY+xfgxStrovInu65efn0xz8+jNuTKG1MaE5dBg9HXMhwI3G/jnBrdQ2vWC4U wlaX4kqSzq3ycy7olD/czoW4lUZcvS9CgI5YBKeOe0sEiLo4hIGFNudWuKU7xx0pHkb78KrKJcp+ h8bRfa2biVRxw4ZoHyNrHI8gbqQbyBB3MAjPL1MeRFmaCk2G2IZx7bVatuSlGwiPtyjsAns19ITU yxu44FVlUzc6ciHKEh/GxosI1a1NaKqiGTT8cAUy+FQUjB3Cg1uhmnDiQvzyEBPqm3tRaF+080sw h7go0QSv1nFWgOjF7mFsvvA6PK1va0TZhJGsmTViOXRDKW7tRtWr0ulujjB0QwhKZWTo+h4uRQUI b+PCfKJfA0aMDerSGHAbXIpaqbqLfTHHuDQITAYoJ4o1xqVx+RpDlMabot9cx6Vh4OY6vKz6zrkZ xrxPx9f438DNESP1yNB1N+NmYJWuQ28ektxKDAJL2YXZZnzkTNQoIrmKMChbUGOCSEwWw4NFeRkG cZ/e/dvDSKsLCKnJXtOqgZ6Dg0TLYFHIW+1j7t0MDjI0QtwoIY6nzEHUWqOi5LC9wsO4c0B4jYOQ gCGmfrgFYthv4GFuvIC41JWq6xp/uotwzaWtBlePP6/uYqzNzrty0rFOD4vYAEbH0fWiGS7FmT7q 6U9RYmY7vl69hWLIhU2z9mGBfngNhAqD+cJiWvFcMn8YEaLgvl7iXHrsDPPuYoUTV7+LYZn0+POV Jj3LRqFJh6Xo4VUUh+ln3AMcvnsCQA1lACA+/yaA4Wc0gJ0BWApANQNAedcEMGtESWlEKWcAm268 BaCcAaAG9wAwmkZi17aUa9dKTNd7I+vY4YeXp5f/3TyMxjoKqdMQ6SBZDkOpwbeMjN6tSw2vJ98a ffP47fu/HL89zjPVdRgC6HgJ4f4eRncKpuEZfrNThbHcKT7NLG+0j7Obbh5ehssepvmLW7j4sNPy NJweFjRChBx+xkZIreIkT4uYlRxe0W0YlyfG1zDdNrJCPSYWgSkiGCnikHSOc5EMsmo3AW64CWjR 2Q1dCNiusLM+jWsHpxYVxSa6XG3RYSEfAAw/owHibFmHybK8dc6sKhXGXefB8bAub7qEapsCun4J cZEcNOL485IV1yZZw8qx6QLiz+gLiMu4wozV1gdVVmY2Zb9h/HJa/PUwrm2abmT4GXcjw4IfuJHh Z2yYHFbAwPN2+Bn7vB0maTEhV6enaonhrOhtW3crvxgQTNz1/faJuy3bYUS5wInLY6Ybij+jW0QM N1QEWt7o6IWvwyxzSkvNLG+0j/MDN0zgR/cYGsGEa7e3NoWqnOlPg+uZ5e1N6avZ3HNYMwONOPyM ZcXgGfFVLf1jjxVxawjthFuzYkDQpgqOl8RJ5iLaYhN1hwU1cEPDz9gbGqcHo6OmpgqJvHnlajfk zWOfDKtjwFGHn7ERZ1iqYuKCkoMKA5NZ2h33HBsQwgwizNkWOJfIVZXl8PyZ4wz5nfB0Dh62yPJc mrr4tj+5yXAF4VXKm6+jj55ytj9Z3mg/PEdnea5hMUt4DZ2hwnwUFcMXLRERdK1jvmyOc8nhVef1 iuvjSpGRo+PPSI6Ol6Dr8EC79UKELmo/jVfnlrfal33XNKe4MS4cCVPBSAphalQOd9EUEUHbUqkl zuUUZL8m6bgqA5p0+BnbpMMlaCsqc+uFxLGOnELx3PJG+8CNWp0fBlDfGgLPotZ1cbzoTqnQMQka JcJ2mQq9nMrUJ1IMsVcbY+2tETg8TBoxJRjmljfaDxWW88NoXKYCLTD+PJHqcJVU8QrGkspoPy+v XJ7D1FNGfVz+Afbjz7jvH2oBytowf1hUBC4OCAp/4tAQY/Uwlrkx0g6zuHZiwJClU7aMQ97bcnUh UrvqxIDhm8OrrW/9/shAf06Gz23H1xt6QHVnDg/s1VFBezOHtS+6Uw8Oc5XQA3EGnJqxJB/ejRge 3n8db2H48vDVsWp9+yW0s9g4sx1frztyZwMRppsYltZALBt+RsaykYGqDP/dykPlfHnqhbH9ggPI m2ulcVjqp1zAaKlsUd9sP5TmpkgyMlgFXoqbeawrryYeQ5l98ONFyX2/A+JDpTt/f7zy8Grrm+8/ jFiKczZlbguv1/2gqe3pDsbB/HgHy4H9LoW8dX6KBGPbhVdd3h4Jmm44kQvuYGY7vt5wBfWZxDAh Gu9gOTm6EAnaU/YDhiVgvxii7NtrN+yBONiPI8XRfjlq3LOPbuxOHB6K/LLSMTLMS/0XfahuThwa ZC/hNTwRFuKXyylFcQ4jc9vx9YY70Ko53cH4DB5bYPk83vcCWZlp3D3GD1WKSt0aReK4W7ZnDs1s x9cbrqBqzMQheIIMd5B8miQyDtJ0rVGTfey98ESRN/dhXMB74jAE7/H7U4F8M7eN/O9VO33/OA4c 7Zdjwgsc9qc4PF55fDW3Xn/Rtb45e/HcFl6vXUGcA4rJi8DzhztIRoFtCxS1dX0JLTBOg0trqvIQ goM26ztI9GBoAyfOufnJuAoXfxNE7IS2dPZcRYVy2XAXydJZikdd1ddwF+OjOLyWUYsWpZY3VBhE PbbCeBejsdBS2DREemhnpo4YLePrziWkH+ljOBsbYWY7vl6lQnTncnIGcOCxEVPOnCyzeNFBOIGy xGi/LFFcpOIkAgP6jjrEFJW3PJDaWCEn+/ExPNqnHskJKscFSdP1gwJgsE+qARJqYaelnMLpKH8L ryGohjnXudZ3iUS+LcXpgTDaVlUM5UmE5KBC22l6MVrGV32rverC+HN4JMMVnG3H16t9GAK6q6c+ HLUQwbCKj+MbFBGFV23tzFDjGX15NA6vpkxDpOd47WyWObOdXq+PraQ7z7MH54F5esKRUgEpxK8e ogl8JwhqE9+fInLf19NTDRoeZrmJTtg2Ymgg0U2OMIbSMCII8+NkQN0pdg6BoH19PX7EVUUPI4AK U6ebYYap2jDEPS0nfxgBwqu1N19Nr8ruHNjmtuPr1f6Myaz2NOsdyr46bpt9a/E3znq7c35xbju+ Xr2COO8/jXPABp5PCfsdwXmjph7Vp9lCsgW2jOiarmpsN41ThtEljFMSI80tIyvR+bKbnvLDvgrh Nc76tdlOWG9MBo4I4VXqJc7l/EW9zq+OCGWYNNhbcWKFqZ4YMVqG1/D7jfbD/PE88p3bjq83MCKM PKcrsHX0inHvhbq0sx457DFChNG28EOPDgBn0wXALXrAs3eeAKr1dXzsMUPp+MxSxQpmXNeBWPMk fSdFXIYxrEXQthokgZuVIdP93LJKJhDcbv+6aJJZnyyWEwJEmIgazHKfeAyyDr2qlhAVCmJoUXVe JQOEGt01Ra7UVXSVK6encGlPw+HFuo7LFa4ujKnHB5g4p1kp60LOlmX6+3dSC5NznC3FCaWIyoD/ +Pjj3v03RV2rIVx9da8PJ9tSnHBGhMst2M4RRtuyOOEsruGt+frhD0tG94VpY47ut/EmRtOxK4ef AeASlZQfLqF79/7le/Wn+k/FA2DEKdIJb3EzqUtpOh0eYmukEeMwLmkb8a7elHdRYgo3dTIV0ysA 7JLbR72IhrDlHibbKq52PNSzUf5+s4QZnXLqjDCWTljr6YSBdEnaR7dryHTtbFmc15BFiKq4ALEz SOy6JQRmDVlcQtaH+HKGCDcuqwxtcekqNm1RFlVR9dIubsQo1I3EAW/TLtsCGXijCqnvlm1h+G2h cWsLR16s2gKzzjK2RaPkbGHgAIFZZxmpFWcAatEW430MbSHPaR1w9OPx16eXw9LLGmNVjF6ffPjr AKFmEOoGiKbXolhCyBmEvBHCLiHKGUR5E4TqV1chZhDiRojVVRQziOI2CL+8irI+Q5T1jRDLqyjt DMLeBuFWV5EhXuhLBE+uvy3rVexMrmXZJ3icjnRqGS90aqx3KV5o1fcLHynNrDnPk6rLzbnqkeSg FdWccY0VpjnDZFRVbhlyNC5elD4etrUMORoXwYvGtq5cNmfJbgtTXurU7aOobCvdiSW11KXmTLRF aMxRwTNrC9yTfYjg5eJRJGYh56ZBjjVSuEVzFrOQM5tRAcRL8/Lx/HaCiCiibqyOz/JhRjWNsUaE 6ecrIy3rnJqNtKYtPM57pFxBGEZ7g/B+3M7jMOyYd4IYfhYx/zEtcr/EihfoDm2vzer2JrovY1NO Q4t4DdPPF29D6rYvx8Twrx9xQ7aHmJI+Pw7hZ11fbM1Gtf0wYQ+8bZ6///rSvn59GaHEDOqsQrlh 4f95Q72Ao4YQqIZwPL4e4jhsImr7Nm6MNb+v2nXloKjs4bbAbrgWpbdd3I6TtfmIxzSdK7o5hjw9 zuav//7fRoy3/vhldh0RpvRNYWPa+cdwjW9f3z/5/zXgnIcLaj5c2GviOAZshyyX8z/HKzlofelK mqfXd/frcuIpAuEjYX8Mf3Tqfw0gigAyJCwHEDmCSALIkGEaQMoRpMSD6KEeMICIAUSYSyA/P79/ /eU4AwlTvr5uh6LAjz+/hL+OIBcbNjq7f19Go8aGqWMg6I/jHweUw0WiXAoGP7Zfe9f4L68v/+sh 7i9Wp+YsU0x4G/cVXF2OGx6aG6DUYGYC+h7+nQOZQg5PnAj08dZ0fkT7vgC81HP0Cp60CTwBeKlk 0hU8k7q+EvBSj8creDZ1fXLAM0XqWXkFr01dnwK8Sx27g+dS16cBD9+/o0xmjWdGBp+eo4cEg5PE 82XvBmf6MvEufrS6hHPRE96aT37OYHupR5NX1LUqZkp/dK9fJtamBpBXMITsACPuEfjm3yes1BQa izWx/9Ka9+cmPE8/tvcWM8k/Dn98//WHw+wmL40jvn5p3rvnBVgRmrkaui78cey1i6E0xaaq7eoh Cr43L+7xp+b9Z/9x/OsIdjGkXqRAGCLEnUTH2yovRYjj93/5LSDNxyVd2Xodd877Mdzxo//5qTsC 0KX22QFSJ6BwtW+AgwgxI05fDIv7f3Tt1/kFyUuxJXy2Dx9eDCvCGLr1dgTqR6THsR0BEBFcpisb g9Vvjz+HMcZj//TsH4FOEhFZ9sGA6PKSA94MBjFeIp5B+2ASwC65881gCsAu+fPNYBrAEI/GfTAD YJfIfzNYBWAED9iC2RFMIZ6w+2A1gGXxAAEukNxOBI8GPpDcWASPBk6Q3GIEjwZekNxsBI8GbqCy uIEAP1BZ/ECAI6gsjiDAE5LZGDwauMLFra1uRwNfSO4viY+54AvJrCgeDXwhmdHDo4EvJJN7eDTw hWSNBI8GvnBxW6Pb0cAXdBZfKMEXkjlmPBr4gs7iCyX4QnLbQzwa+EKyBogfMoAvJEtgeDTwhWQ1 DI8GvpAsVeLRwBeSVUs8GviCyeILEnwhqUPBo4EvmCy+IMEXkvV3PBr4QrIUj0cDX0hW5fEjXvCF KosvKPCFKosvKPCFKosvKPCFKs9MAXzh4pamt6OBL1RZfEGBL1RZfEGBL1SXfOG5jJPrfj3NLYak AvzRe7eY5lqCOzTj5fUfP/cfgHJJFvKb/9m/HB9Xl+WlGjO+H8OfB5wqiXPhamrb9RIa67efX58B BelGEaUElLd33/tj9xPM3CuLdKIklgAspAslsUrAQjpQEksCFtJ9klgKsJDOk8TSgIV0nSSWASzk QySJNTphVSN9JollAYvA+A1WDVg5eA/z9epimvdmMGA+ptKxDwbUv5g0vhkMuH8xa3wzGJAfU4DZ BwP2X0xB3wwG9MeUcvbBRv5PEgMmmAWwHA4A03SLqeDsR9gCwHJ4AEzSbZEl9pcAlsMDYIpukxvK o8EUgOXwAJig2yKHB8D83BY5PACm5zZ5JgQaDDwgeToEGgw8IHlyB3pcAB6QPMMDDQYekDxfBQ0G HpA8aQUNBh4gcngAzMutyOEBMC23IocHwKzcihweAJNyW+bwAJiT2zKHB8CUfNQWcsFgRj4twGOC gQeUOTwA5uO2zOEBMB2fJK1MMPCAi0Xmm8HAAy4Wmm8GAw/AVpvTYOABFyvOO2ACwL6+fXpv3FQe tthacwoKOIatNKeggGHYOnMKCviFrTKnoIBd2BpzCgq4ha0wp6CAWdj6cgpq4hWB8WsoiKvY2nIK CqIqtrKcpCjQHVtYTmIB37Fl5SQWEB5bVE5iAeOxJeUkFlAeW1BOYgHnseXkJBaQHltMTmIB67Gl 5CQW0B5bSE5iAe+xZeRkPAXeY4vISSzgPbaEnMQC3mMLyEks4D22fJzEAt5ji8dJLOA9tnScxALe YwvHSSzgPbZsnMQC3mOLxkks4D22ZJx8+APvsQXjJBbwHlsuTmIB77HF4iQW8B5bKk5iAe+xheIk FvAeWyZOYgHvsUXiJBbwHlsiTmIB77EF4iQW8B5bHk6OVIH32OJwEgt4jy0NJ7GA99jCcBILeI8t CyexgPfYonASC3iPLQknsYD32IJwEgt4j623RqxxjYF7e4zLYx6Pzcfnv04zR0rBNAUHLKNUTlNw QDRK8TQFB1yj1E9TcEA3Sgk1BQeMo1RRU3BAOkohNQU38Y4Qb1NwEHIp5dQUHERdSkU1SWNwC0pV NYkHfkGprCbxwDEoxdUkHngGpb6axAPXoJRYk3jgG5QqaxIPnINSaE3igXdQaq1JPHAPSrk1iTf6 R02puCbjcgF4mfwDZqM1pe6axCsBL5N/wKy0plRfk3gK8DL5B8xOa0oNNolnAC+Tf8AstaZUYpN4 FvAy+QfMVmtKPTY50AD/oJRkk3jgH5SqbBIP/INSmE3igX9QarNJPPAPSnk2iQf+ga/QusJPeL89 PT92z755AVCARLvIdUi0l1yHRDvKVUh85fY6JNpdrkOiPeY6JNpprkOi/eY6JNp1rkPm9x58jfc6 ZH7vwdd7r0Liq77XIfN7D74KfB0yv/fgq8LXIfN7D75KfB0yv/fgq8bXIfN7D76KfB0yv/fga8rX IfN7D77CfB0yv/fg683XIfN7D776fB0yv/fga9HXIfN7D74yfR0yv/fg69TXIfN7D75qfR0yv/fg a9jXIfN7D76ifR0yv/fg69vXIfN7D77a7QoHFbHf3v1pY5waX+peA0GGBV/nXgNBagVf5F4DQU4F X+FeA0EyBV/eXgNBFgVf214DQXoRX9heA0FeEV/VXgNBQhFf0l4DQSYRX8/eEBKoja9mb5CA2/ha 9gYJyI2vZG+QgN3YZc0JJKD3xSXNtyEBv/G19Q0SEBxfVt8gAcPxFfUNElAcX0zfIAHH8XX0TawE juNL6Bsk4Di+er5BAo7jC+cbJOA4vma+QQKO48vlGyTgOL5SvkECjuNr5Bsk4Di+Or5BAo7j6+Ib JOA4viK+eYwDx/G18A0ScBxfBd8gAcfx9e8NEnAcX/neIAHH8TXvDdLAcVHgq90bJANIbI6PesWA xOb4qFYMSGyOj1rFgMTm+KhUDEhsjo86xYDE5vioUgxIbI6PGsWox2AjKUBic1wBx/EV6w0ScBxf q94gAcfxVWoHm7H+9vj15fnp5bN3j+69eXp5BF7h69S7iMAvfKV6FxF4hq9V7yIC3yjV6h1E4B2l WL2DOPGP4BM7iBMPCb6xgwh8pFSqdxAh9lIK1TuIEIMpdeo9hoPTUOrUe5DgNZQ69R4kuA2lTr0H CX5DqVPvQYLjUOrUe5DgOZQ69R4kuA6lTr0HCb5DqVPvQYLzUOrUe5DgPZQ69V40B++h1Kn3IMF7 KHXqPUjwHkqdeg8SvIdSp96DBO+h1Kn3IMF7KHXqPUjwHkqdeg8SvIdSp96DBO+h1Kn3IMF7KHXq vZELeA+lTr0HCd5DqVPvQYL3UOrUe5DgPZQ69R4keA+lTr0HCd5DqVPvQYL3UOrUe5DgPZQ69R4k eA+lTr0HCd5DqVPvjdLBeyh16j1I8B5KnXoPEryHUqfegwTvodSp9yDBeyh16j1I8B5KnXoPEryH UqfegwTvwZZrGxfPfYiQX/yx+Xhu2sdP769f34aCLUAiSXQLJJJEt0AiSXQLJJJEt0AiSXQLJJJE N0BiS723QCJD8C2QyBB8C2R+78GWhG+BzO892CLxLZD5vQdbNr4FMr/3YAvJt0Dm9x5safkWyPze gy023wKZ33uw5edbIAneU16BJHjPNUiC91yBxJaob4EkeM81SIL3XIMkeM81SIL3XIMkeM81SIL3 XIPM7z3YUvctkNm9R2CL37dAZvcegS2H3wKZ3XsEtkB+C2R27xHYkvktkNm9R2CL6LdA5vcebFn9 BsiL5bjjt+//cvz2OIMs69Lp4fS5H4/fPsFZK4/+5RjPtR7wLpHyt6fn10/f/2V+ibZwfdtewLvE yC9f3r7/y5cvE54vpAnjFqfiOPzH8McU3iU6Jo52tWFK7IezXH57fHpxT+++Oz52ry/Ov3x4gEQe OxsgjQTI56effTzF/tH5j+P766+AiDx/NiKWa8TVRV5iYxrSW4DsfvLd57fXp5dwmU8fXQNHmF88 HP7iqb1d82VQVSmM28ULE5X15XBIvXt98X8tAAV7kndE6U4oo35NYZxrQpHnaykBBXtqd0Q5X4sE FOxZ3QFFna9FAQrimXNCOV+LBhTsOe4BRZ+vxQAK4slyQjlfSwUoiIfJhGLO12IBBXuSfEARw7U0 cfeGAURjZisTyDB5+DF40MuLH51IXHSiD//Xj2NzXKZYpRy2u/ox/PGrf+l8cNtidEh5CeuyQ77/ +nZ8HW5MXswjjx9cnHYlms7avlijXPLJFcpwX1Uhh7I+oBze/fHr+8vHYfRxeXF90x6eTeAJwLvk p3t4LoFXAt4lj93BE2UCTwLeJd/dw6sSeArwLnnxHl6XwNOAd8mfd/BKkcAzgHfJs/fwTAKvArxL Pr6H1ybwLOAhvGLCG89uW+HVgEfwD6lTfAYHuZgT3wNsUoDgIRcz4nuAfQoQXORiPnwHcHyorQHB Ry5mw/cA6xQgOMnFXPgeoE8BgpdczITvAOpUFISlO/JiHnwPMBkGwU8uZsH3AFNxEBbyyIs58B1A kwqEsJ5HXsyA7wGmIiEs65EX8997gKlQCKt75MXs9w5glYqFsMhHXsx97wGmgiGs9ZEXM997gKlo CEt+5MW89w6gTYVDWPkjL2a99wBT8RAWAMmLOe89wFQ8hHVA8mLGew8wFQ9hOZC8OILcAaxT8RBW BcmL2e49wFQ8hMVB8mKuew8wFQ9hjZC8mOneAWxS8RCWCsmLee49wFQ8hBVD8mKWew8wFQ9h4ZC8 mOPeAWyTA0PwlIsZ7j3AVDyEjc/lxfz2HmAqHsLu5/JidnsHsEvFQ9gCXV7Mbe8BpuIh7IMuL2a2 9wBT8VBNKQiCp7hUPIQd0dXFrPYeYCoewrbo6mJOew8wFQ9hb3R1MaO9B5iKh7BBurqYz94B9Kl4 CLukq4vZ7D3AVDyErdLVxVz2HmAqHkY1VDAR9Bn7J/865tAkJhser0vURvaDc3x6PB9OLdGpuAHG DjBf3wADm4iLGNV4Ke71l5cxjVGmGuX/uymN8fzaDXntokxmMf7vxHtpkMN3xR8AKHVPGCABQMkM BgaonIBSoQEDJCegVEjAAKkJKBUKMEB6AkqFAAyQmYBSro8BqiaglMtjgOwElHJTDFA9AXGZLSZq J3MQKKSJ28nkAwppIncy64BCmtidTDegkCZ6J/MMKKSJ38kEAwppIngys4BCmhieTCmgkCaKJ3MJ KKSJ48kkAipWThxPZg9QSBPHk2kDFNLE8WS+AIU0cTyZKEAhTRxPZghQSBPHk6kBFNLE8WROAIU0 cTyZDEAhTRxPZgFQSBPHk9N/1GN84nhy3o9CmjienPCjkCaOJ2f6KKSJ48kpPgpp4nhybo9Cmjie nNSjkCaOJ2fzKKSJ48lpPApp4nhy/o5CmjienLijRpgTx5MzdhQScFwmp+oopHJC4nJcyQmJPRJX ExKX40pPSFyOKzMhcTmuIsfD74I+NXx6OQ4KoEIkl7HddDUfvzzpHw5983E8DNtFAh7Z7wKe+WGA Ovz1q//qAY/sMyPe//XUvMwvsEyuNbsVUPxwePHHp493KMSr5NYaN6E9vf9VFT8cfn56Pz69Pr51 TxMi2bUjokggJrfWuBmxnCNCOi65tcbNiDKFSHb5iKjmiJDhS0r5bkbUKURyCIiIJoVI5mJADF3d HD+3buplDhNF6OW3jy8TFJmCwUUCp78278fhZRDbmWQuDeFxb/3hw7+4MeaNzFvFvP9xU8xzz2NM Se/K9T++vgz//tt//38PKYc5Yfxw6N6+FoCUosQcKeUoSyQBSKmnwhwp5SBLpBKQUhF4jpS6+yWS BKQUPedIqbtfIilASlFgjpS6+yWSBqQUN+dIqbtfIo3p6yKpcJwjpe5+iVQBUqqX50ipu18iWUBK 9fIcKXX3S6QakK5xPHn7CyTQ0qS34VpA7bB8BgUsT+oeF1A7NJ9BAc2T4scF1A7PZ1DA86QCcgG1 Q/QZFBA9OR5YQO0wfQYFTE8OBBZQO1SfQQHVkyOABdQO12dQwPXko38BtUP2GRSQPfnMX0DtsH0G BWxPPuwX0fMq20EPk94+awF1le2ghElvm7WAusp20MCkt8taQF1lO6hf0ttkLaCush10L+ntsRZQ V9kOipf0tlgLqKtsB61LejusBdRVtoPKJb0N1r81LxPa7n8zKGB7cvurxVVdZTsoW9LbXi2e8FfZ DpqW9HZXC6irbAc1S3qbqwXUVbaDjiW9vdUC6irbQcGS3tZqAXWV7aBdSW9ntYC6ynZQraS3sVpA XWU76FXS21ctoK6yHZQq6W2rFlBXYztoVNLbVS2grrId1CnpbaoWo9CrbAddSnp7qgXUVbaDIiW9 LdUC6irbQYuS3o5qAXWV7aBCSW9DtYC6ynbQn6S3n1pAXWU7KE/S204toK6yHTQn6e2mFlBX2Q5q k8N8Jnn6+T/+28fw4y/N03F4a1pLV7ayVu3gbT8+vTwdx9noah56WChWmq/u6fj4y3p1iHNRJjL8 cbyQJMj/gAu5OKX97N9f/DNMjC5prj5+ad6eXtZqE6UGAVL849u4IrBAb76lXefEIN166h+fXh+P f/0E572gN92KUMUKCrwNu9lWgPJ+BQXeht1kK0K5FRR4G3ZzrQjVraDA27CbakWodgUF3obdTCtC NSuoyduQ64AjVL2CgmcLdvOsCGVXUPBswR7uFKGqFRQ8W7CHOkUos6Yo0B17mlPE0mss4Dt2X7CI pdZYQHjshmARS66xgPHYncAiVrnGAspjtwCLWOtIM82VsXt/RaxNqAHSYzf9Clj9OtZMs2Xsbl8R ax1spukydpuviLWONtN8Gbu/V8Rah5tpwozd2CtirePNNGPG7ugVsdYBZ5oyY7fyiljriDPNmbF7 eEWsdciZJs3Yzbsi1jrmTLNm7K5dEWsdc6ZpM3a7roi1jjnTvBm7T1fEWsecaeKM3aArYq1jzjRz xu7MFbHWMWeaOmO35IpY65gzzZ2xe3FpNx3bN8MC3mM34YpYmwEO8B67+1bEWsecafqM3XYrYq1j zjR/xu63FbHWMWeaQGM32opY65gzzaCxO2xFrHXMmabQ2K21ItY65kxzaOyeWhFrHXOmSTR2M62I tY450ywau4tWxFrHnGkajd0+K2KtY840j8bumxWx1jFnmkhjN8yKWOuYM82ksTtlRax1zJmm0tgt sgKWWMccBbzH7o0VsU4xp3t96Z8+zWaOArsvVoQbw87Ha398ev/rCQjPe9EugAQA4UkvmgVQCUB4 xot6ASQBCE93YRdACoDwXBfVAkgDEJ7owiyADADhWS70AqgCIALF1QLIAhCB33IBNAZi9J5VEahc EhKojT0KKiKJJRJwG3sUVEQqlkhAbuwRUGFIJJfNBBNNgT36KSKt2gnojT3yKSKt2gn4jT3qKSKt 2gkIjj3iKSApv0QChmOPdopIbokEFMce6RSRljEXJpcCe5RTRFoGXZhaCuwRThFpFXWB49ijmyLS MuzCtPLyFnE7SMu4C5PKy5vD7SAtAy9MKS9vC7eDtIy8MKG8vCHcDtIy9MJ08vI+cDtIy9gLk8nL +7/tIC2jCkwlL2/7toO0jCowkRTYI5ci0jKqwDRSYE9aikjLqAKTSIE9YCkg6WVUgSmkwJ6rFJGW UQUmkAJ7nFJEWkYVmD4K7ClKEWkZVWDyKLCHJ0WkZVSBqaPAnpkUkVaDOeA49qikiLSMKjBtFNgT kiLSMqrApFFgD0aKSMuoAlNGgT0PKSItowpMGAX2GKSItIwqMF0U2NOPItIyqsBkUWAPPYpIy6gC U0WBPesoIi2jCkwUBfaIo4i0jCowTRTYk40CkllGFZgkCuyBRsF9G0javHx57N+9ny80wJ5lFNFG 3xsrqotFC1imB6zR+5qnV/f4+an7vIDD0j3AjS7ofO/f3717/Pry5fXrsHABILG8D5CjLz4/vXz9 9vnt6TF2xS/TFaIp27TjDb91T48/vS3uFk3aph3v9vOwzuOxO35b4KEJ17SQX3oxW57gC49NK5ct 9/HT6/sxtN2UVcAXIPchIa7hC5H7kBDg8PXIfUiIdPiy5D4khDx8dXIfEmIfvki5DwlBEF+r3IeE aIgvWe5DwqMfX7nch4QxAL6AeZXq+DrmdciM3jNBZvSeCTKj90yQGb1ngszoPRNkRu+ZIDN6zwSZ 0XsmyIzeA1EdXw29Dpn/2YOvjV6HzOg9E2RG75kgM3rPBJnReybIjN4zQWb0ngkyo/fAAAZfU70O mdF7Jsj8Izd8ofU6ZEbvmSAzes8EmdF7JsiM3jNBZvSeCTKj94xj9RJflr0OmdF7JsiM3jNBZp/3 lPjy7XXIjN4zQWb0ngkyo/dMkATvKZeQz68vn84z/BJf791FHIeCJb7wu4tYAiLBdXYQJSASPGcH EXoGXxDeRdSASPCbHUQDiAS32UGsAJHgNTuIFhAJTrODWANiPp8BRUKJryDvQ4LX4EvJ+5DgNvia 8j4k+A2+uLwPCY6DrzLvQ4Ln4MvN+5DgOvi68z4k+A6+AL0PCc6Dr0TvQ4L34EvS+9EcvAdfm96H BO/BF6n3IcF78NXqfUjwHnzZeh8SvAdfv96HBO/BF7L3IcF78BXtfUjwHnxpex8SvAdf496HBO/B F7v3Ry7gPfiq9z4keA++/L0PCd6Dr4PvQ4L34Avi+5DgPfjK+D4keA++RL4PCd6Dr5XvQ4L34Ivm +5DgPfjq+T4keA+6jH5hlA7eg66lX4AE70GX1C9AgvfglybvQ4L34Jco70OC9+CXKu9DgvdQqvV7 kOA9lKL9HiR4D34J8z4knHRCST+Mepj+6f3LL837XAkgKVPwURMDh1nAOUCUifcCB47/oUy3Fzhw 6g9lkr3AgcN+KFPrBQ6c8UOZUC9w4GgfyjR6gQMn+lAmzwscOMiHMmVe4ACrKfPkBQ4c20OZHC95 CISmTImXQMBoykR4CQSUpkx/l0DAacqkdwkEpKZMdZdAwGrKBHcJBLSmTGuXQMBrymR2CQTEpkxh l0DAbMrEdRkZgdmU6eoSCJhNmaQugYDZlKnpEgiYTZmQLoGA2ZRp6BIImE2ZfC6BgNmUKecSCJhN mWgugYDZlOnlEgiYTZlULp/VwGzKVHIJBMymTCCXQMBsyrRxCQTMpkwWl0DAbMoUcQkEzKZMDJdA wGzKdHAJBMymTAKXQMBsytRvCQTMpkz4lqNHYDZlmrcEAmZTJndLIGA2ZUq3BAJmUyZySyBgNmX6 tgQCZlMmbUsgYDZFtb0EGpmtsAOtuhHlKCf/+fjij8Xh/dtfp935sWOtiGXmWMczFnZMErFgDcuv H0f/5XGYiT5OaEiS76DB7vnY0ckOGuxKjx2i7KDBwZjYccoOGpyKiR2s7KDBkZjYEcsOGpyHiR22 7KCBH2DHLjtoFtCQYX4HbYz1CjuK2WMvOAN2LLMHB96AHdHswYE7YMc1e3DgD9jRzR4cOAR2jLMH Bx6BHenswYFLYMc7e3DgE9hRzx4cOAV27LMHB16BHQHtRWHwCuw4aA8OvAI7GtqDA6/Ajon24MAr sCOjPTjwCuz4aA8OvAI7StqDA6/AjpX24MArsLt07sGBV2AXzO3BgVdgF8vtjSbAK7AL5fbgwCuw i+T24MArsAvk9uDAK7CL4/bgwCuwC+P24MArsIvi9uDAK7AL4vbgwCuwi+H24MArsAvh9uDAK7CL 4PZGxeAV2AVwe3DgFdjFb3tw4BXYhW97cOAV2EVve3DgFdgFb3tw4BXYxW57cOAV2IVue3DgFdhF bhFOzeGcf25+BVCAJHjGFUjsIrdbIAkecg2S4CXXIAmecg2S4C3XIAkecw2S4DXXIAmecw0yv/dg F7ndAIld5HYLZH7vwS5yuwUyv/dgF7ndApnfe7CL3G6BzO892EVut0Bm957h2ZkbMrv3aOwit1sg s3uPxi5yuwUyu/dorMrsFsjs3qOxi9xugczvPViN3S2Q+b0Hq7y7BTK/92D1eLdA5vcerErvFsj8 3oPV7t0Cmd97sIq+WyDzew9W5XcLZH7vwSr/boHM7z3YCnVX9nBYQ/PePb69f30Zz7TWycno4njJ vbP7nG/c8+d3/wFAyHaLVzTuq9d0b0/DzUGRW2PniSkoAVCEdlpDlQCFVyp044aG//Z//j8zPblB D5hsPPcmAv32+PL1+fnx6eMDkAht7hdIL0dAwlLUdq4EpHcXrwg6z6BHRVskAUjYcL5FKgEJG8W3 SBKQsMF7izSmDw16xLNF0oCEDdVbJANI2Ai9RaoACc3x6TjAEenl+FhMjEKPZ1JYwCn0QCaFBaxC j2BSWMAr9NAlhTUxi8D2DdbELQLfiyWWmPoRPUpJYUE/oocnKSzoR/S4JIUF/YgekKSwoB/RI5EU FvQjeghip4MwT1jlqR/xvN9iTf2I5/0Wa+pHPO+3WFM/4nm/xYJ+RCsAU1jQj2j9n51OIT1hyakf 0eq/FBb0I1r7l8KCfkQr/1JY0I9o3V8Ka+pHAu83WFM/EnjfLbHUqR8JvN9gTf1I4P0GC/oRrfZL YUE/oqV+KSzoR7TOL4UF/YhW5dnp/N0Tlp76ES3JS2FBP6IFdCmsqR8J/NpgTf1I4NcGC/oRrZtL YUE/okVzdjr8+LfHX97n8x+0Xm6LBH2IlsptkaAH0Sq5LRL0H1ogt0Waeo8QTVdIU98RuL5CgvkP WhG3RYL5D1oMt0UaBUQGrYPbItWAxOc4KK8NWv6WgAKWo6VvCSigOVr2loACnqMlbwkoIDpa7paA AqajpW4JKKA6WuaWgAKuoyVuCSggO1reloACtqOlbYnoCWxHy9oSUMB2tKQtAQVsR8vZElDAdrSU LQEFbEfL2BJQwHa0hC0BBWxHy9cSUMB2tGwtAQVsR8vVElDAdrRMLfGEB7aj5WkJKGA7WpaWgAK2 o+VoCShgO1qGloACtqPlZwkoYDtadpaAAraj5WYJKGA7WmYWoOol1E/TyBitL0thAbPQwrIUFlAL rShLYQG30FKyFBaQC60hC1h2hjXPoaPFYymsqe0JpN9gTW1PYP0Ga2p7Au03WFPbE3i/wRo9u0Ir wwJWtcSacugVWhKWwhKAReD9BqsELALvN1gSsAi832ApwCLwfoM19SOB92aJVZ76kcD7DdbUjwTe b7CmfiTwfoMF/UiocG6xoB8JNc4tFvQjocrp9RJryqFXhDrnFgv6kVDn3GJBPxLqnFusqR8JvN9g Tf1I4P0Ga+pHAu/VEkud+pHA+w0W9COhzrnFgn4k1Dm3WNCPhDrnFgv6kVDn3GJBPxLqnF4usaYc ekWpc26wpn4k8H6DNfUjgfcbrKkfCbzfYE39SOD9Bgv6kVLnLJdY01yhotQ5N1jQj5Q65wYL+pFS 59xgQT9S6pwbLOhHSp1z0nz0UL84tT2B9xusqe0JvN9gTW1P4P0Ga2p7Au83WND2lDrnBgt8iFLn LJZYp7kCpc65wYJ+RG9mksKCfqTUTDdY0I+UmukGa+pHAu83WFM/4nnf+yXWaa5AqL9usaZ+xPN+ iwX9SKi/brGgHwn11y0W9COhArvFgn4k1GB7t8Q6zRUIVdgtFvQjoQ67xZr6Ec/7LdbUjwTeb7Cm fiTwfoM19SOB990S6zRXIFRkt1jQj4Sa7BYL+pFQlN1iQT8SqrJbLOhHQll2iwX9SKjL9u0S6zRX IBRmt1hTPxJ4v8Ga+pHA+w3W1I8E3m+woB8JxdktFvQjoTrbNzMsWLdQEUqz/ZRv784rKSpCXba3 M5zT9RC4PuUbn1674/ySCFQ3S6jTVRGYPmVdju9PX2b6popQld1igdcQyrJbLPAaQl12iwVeQyjM 9mqBFVvetT8UP3x27Z/8y9G//+mteXnq/nI4NM/Px/em8w8P/zP88/Ty6dC9fvnSvLjDZ//+4p8P b0/uUByO4TVcSxxgHt30ZfH7RG1UY4qHj+4n7x4/fnk6dj9994dDczzM3/lj8U275s/9e/PFz63L uolLgB6+PC1MT78GO1HJHbsyfOuz929/XX7t/K1oX6gde3OyPz598e6X5um4gDi9G1BUuwPSFA+P w+dH0/HnYFC2Lm0Ruubh45fm7c2/w7eNvwSbSu+YNGMLHZv349e3cxONvwdDWe7cY98XD+3Rf4Mb G36MFydv7O4wED53ty/KVtbGFSWlu30hvLGmch7X3ZNdpAmlu0/2od2/DJ9+/Hh7enn8aKFRVm9G DOV2MLri4dOwdu6r/+ofjz+9+8Y9Pr++Qqek/3bxqkIH9a/vnx/9t4l9p1+DnS3TZr0Es+C7X95e n59e/Mz4/GaA8PsI33///SF89u1Q/OfD+9Pb4V+HHz6mH9rhh0P41K1sKbds6eqCwZaOyJaOyZYu A1u67GzpaGzp2GzprrHlm42PhhNnvpVxM+v4sECQRybIIzydPJWgkacSPPIEezZ5AkZm8gRECnkq wSXPiJA91KgtW9qmY7DFENlimGwxGdhisrPF0Nhi2Gwxd2GLTrBFNgy2tES2tEy2tBnY0mZnS0tj S8tmS3sXtpgtW5rO0tliCxpbbMFjS7BnsyVgZGZLQKSwxRZctowIZ7Z8M31RlF1vT7T55rTzqmrD hLaxjaya8wCnaUTTF6IL87fO9vOxjm36qq6sEZWrvDUKw7Rqw7RKcwbMVhOZpplM0xmYprMzTdOY ptlM03eJSzbBFs4I2TZEtjRMtjQZ2NJkZ0tDY0vDZktzF7bUW7YozgjZ9kS29Ey29BnY0mdnS09j S89mS38PtpgiwRbOCLlWNLbUiseWYM9mS8DIzJaASGFLrbhsGRFmYx7hS6mqSpzHPHXnmkI3jXOq avqyO495Klkq0Uld6y4MgbU4j3mUL0VVFk7atq6rymOYtk0pV5Izuq5rItNqJtPqDEyrszOtpjGt ZjOtvktc2qaUK6lJ9Sa4TE9ki2eyxWdgi8/OFk9ji2ezxf8OKWWzTSlXZewGKnkaSSNPI3nkCfZs 8gSMzOQJiBTyNJJLnhEhe6jZppSrsmLUOhtLZItlssVmYIvNzhZLY4tls8XehS2blHJTSM6AOYy/ aGxxTLa4DGxx2dniaGxxbLa4u7Blk1JuipIz6G1LGlvakseWtuSzJWBkZktApLClLblsGRGys2WT Fg5s4Qx6W6LqpmWqbtoMqps2u+qmpaluWrbqpr2qulkNejGc2SSHm0JwxrotUXvTMrU3bQbtTZtd e9PStDctW3vTrrQ338rrUyWBnihtcsWBPJyhb0fU3nRM7U2XQXvTZdfedDTtTcfW3nRXtTf0gFNt MsaBMwWjdtkRFTgdU4HTZVDgdNkVOB1NgdOxFTjdSoHzTRZtXGR9Q4oGUExZ1+UpAv2X/zJ+fbH+ stPHHlz7OGjQHz/8c//4y/tMHJ380x+jvHoNV3aNl0KFbvgcjN79IHMfQWZvRNNux9QVD90oh4c+ m36Lrb1vM2uq7pxV7+E/KwpRV/2s0RYAutm23/iXuFF3bL/h2gNd2l+P/uPUKOe3wsU1m7Y9QUTz T/74+HPz/NWfrE/vBGMpdoyrwbhtus9DD5yMT+9E8rp6x7qfdetI+FWHjm/GC9hEgxGiEsXDY/j4 xzF8IRiFDhlhEn8IUP3GOQGqHa4G4tnpQuD3eA3FjqUdW+GjC/16DK7nu5P17L3YEs0ORWoxRwhU nBFz+35AUhvHBiA9NenbvCkjO/1OH9Z+9IazzeezUak34XS0agNtzhaTA3Q7Pd1pvNtEm3mESbhN 0datrssdt3HlntuEvwxuE290FQKmBpY7DRxMH34e1sWMNuPPsW/NTkv5cB8zg+nzau/jkYbTOUXQ h9Ovw3XteAL6STCacZ4EZ4R5eAvDaqsrde4u29teVlKVtXKdFeLcXWXMS3nVNZX2rSncubuMk0Xv OtEZo6O0buiuf/mXfzn49/fX90OMCnHw0b+/fjk0zoW2+TisHhn2ED6f87FhSvJj46Ygu14+9M8g +88g+88g+88g+/sG2RsndgmBhqkYmaSOKJbvmGL5LoNYvssulu9oYvmOLZbv2jXTtFWLid03iAqi MJ3QvbPbLPb5I03tG7ue420Qlhcys7uVjAn9h+EsQHVELb5javFdBi2+y67FdzQtvmNr8V1xx8zU RvbRFIaTCndEVb1jqupdBlW9y66qdzRVvWOr6p1ep8JF2TpZbwOYLEXRWdWtsy7xL0XXxmbdRLTw l76JIq10RDtBriPaye5Wdm5kJk0804HBTqKK3zFV/C6Dit9lV/E7morfsVX87qqKnxHREmITw8m1 O6KW3/VMzmTQ8rvsWn5H0/I7tpbfrbT830RW1iREJ5ozkPdETb9navp9Bk2/z67p9zRNv2dr+r3a iTTnQXYlhUqP301fxr2EtSmNq7Tye+N3QDh9/9ruVg4mpCyaM373RLW/Z6r9fQa1v8+u9vc0tb9n q/39XdT+VULEojjrYT1R7e+Zan+fQe3vs6v9PU3t79lqf39V7U9iS0K1ojjrYXuivL9nyvv7DPL+ Pru8v6fJ+3u2vL+/i7zfJvQqkrMetifK+3umvL/PIO/vs8v7e5q8v2fL+/u7yPvtJgneFA1n7NwT 5f09U97fZ5D399nl/T1N3t+z5f39XeT9dpOlDmxhjHLrgibvD3YstkR7LlsiRl62REQCW4IZky2A kJ0tifx0zRjl1gVN3h/smGzhy/sjRm62kOT9wYzNlrtsqmkT+eKaMcqtC5qwP9gx2cIX9keM3Gwh CfuDGZstVzfVJLElkSm2jFFuLWhK/mDHY4vgK/kjRma2CJKSP5hx2SLusoumTWSILWMRay1oGv5g x2QLX8MfMXKzhaThD2ZsttxlF02byOVWjEWstaAJg4Idky18YVDEyM0WkjAomLHZshYGzbRnw8cG PU8vunRFQe1KgEaT01eqW3mWyAJXjOWvdUnT/AQ7Hs9KvuYnYmTmWUnS/AQzLs/KO2p+7DYXLErO KLmkaX6CHZMzfM1PxMjNGZLmJ5ixObPW/NRGqjo8dDYhqqlVV9qu2Gp+dO0bpdvNSrVoU6mqb+2O 5ucEudb8nOxuZGe9zT0LwRmVlzTNT7BjspOv+YkYudlJ0vwEMzY777JzZ73NPQvBGZWXNLVPsGOy ha/2iRi52UJS+wQzNlvWap+6VlrK65qfCSRQI1jEnV0vLqw9fexWvm2z11HrSOebpOmEgh2Pb5Kv E4oYmfkmSTqhYMblm9zTCfGi0zZ7LQrO6FzSFD3BjskWvqInYuRmC0nRE8zYbLmLoqdOZK97hha/ ljRFT7BjsoWv6IkYudlCUvQEMzZb7qLoqRPZ656hja8VTdET7HhsUXxFT8TIzBZFUvQEMy5b1F0U PXUie90zVPG1oil6gh2TLXxFT8TIzRaSoieYsdlyF0VPvc1ei5ozylU0Rc+wMT6LLXxFT8TIzRaS oieYsdlyF0VPvc1Bi5ozytVERY9mKnp0BkWPzq7o0TRFj2YrevRdFD11IvtsOaNcTVT0aKaiR2dQ 9Ojsih5NU/RotqJH30PRI4pENthyRrmaqOjRTEWPzqDo0dkVPZqm6NFsRY/Gb9WJYU0iK2w5o11D VPYYprLHZFD2mOzKHkNT9hi2ssfcQ9kjikROVzL067UhKnsMU9ljMih7THZlj6Epewxb2WPuoewR RSKnKzn6dUNU9himssdkUPaY7MoeQ1P2GLayx9zjfNxwkwmtBUe/XhH1ORVTn1Nl0OdU2fU5FU2f U7H1OdVVfQ6JLducbuk4o92KqMypmMqcKoMyp8quzKloypyKrcyp7nHGrSi2Od3ScUa5FVEpUzGV MlUGpUyVXSlT0ZQyFVspU+0pZcYP2UJIX3qTVJju6Esng1uZlcj/dpwRcUVU1VQ9k1kZVDVVdlVN RVPVVGxVTXX1PFzGnDuRBe4442JLVMZYpjLGZlDG2OzKGEtTxli2MsbeQxkjikQWuOWMiy1RGWOZ yhibQRljsytjLE0ZY9nKGHtVGUOPMCKRC245unVL1MdYpj7GZtDH2Oz6GEvTx1i2Psau9DGRIbUU 8812yyK8V8pxb5Juvq2zdS6Mb/rxT2cVqJTDQod+MsPQLJE8bjgC9JoorKmZwpo6g7Cmzi6sqWnC mpotrKnvIawRIpE8bjgC9JoorKmZwpo6g7Cmzi6sqWnCmpotrKlXwppvYogoar7XvK1CeFFj6LFy FpPaGHeKMQa1zTkqOTGidM34dwzPtmnn8MWM4XVNlOTUTElOnUGSU2eX5NQ0SU7NluTUbv3wc2K+ YkvGJ5iKQybbj0u5gGQjk8pu/FfUZ5aJkZGtGUgmMCTbZqtlwRmVN0QlT8NU8jQZlDxNdiVPQ1Py NGwlT1OuSVbaOcmsdueAdabY+HulbfyTmA2uxlWktRj/RRyYIcQ2wy0Lzhi+Iap/Gqb6p8mg/mmy q38amvqnYat/mmrNsFbHyDQLZAOZ+mok03zpqfAjkcJkTi5H8eP7upvIiSFaIjnec0bxDVE41DCF Q00G4VCTXTjU0IRDDVs41FzdCii5IPD8n8eQaJsHD5GUMbhviTqilqkjajPoiNrsOqKWpiNq2Tqi 9i46IrHNgIchBEM13xJ1RC1TR9Rm0BG12XVELU1H1LJ1RO36lF+nY4RR5WwApZrwjhifcVLPnm3j 1lFxqc3wBJuP0ofxu4bxu8UQbZs8Lz1HcN8SJUgtU4LUZpAgtdklSC1NgtSyJUjtenMh28eZn5nz rD7zbD4XDLRQNvxNzylmfVOchvUIfpXbRHvpOBL9jiha6piipS6DaKnLLlrqaKKlji1a6laipW+y OOXNgWAxmz6ltLp+kW+ozyl4MaYogGalPv+llBbzyCy3uXbJ2oKxIwqeOqbgqcsgeOqyC546muCp YwueuvVWRF1hvSnL2XxQDGVg77Uvy9Agdp4+LSGzoOY083rMZ8W/+AJHs22SXrL2buyISqmOqZTq MiiluuxKqY6mlOrYSqmu2WQdYn5BzB6YZUweDM/H0ppqPTBrYfjVlrP5opwswvteYPKnZSJJz9r1 sSPqprqeybMMuqkuu26qo+mmOrZuqlvvRtQWTa/0IoWqiiGzJSPR5inU8VEqxaZADclTdJK+TCTp WVs+OqLQyjGFVi6D0MplF1o5mtDKsYVWTiUrjnU3I1nnpvRp4JSaPTL75jxu6/V6lmkqQga1TKTq OUfD1o4o0XJMiZbLINFy2SVajibRcmyJlqvXs4Axy+7Wufqmmh6EZ6Lp+GtbGDUM+mc1IVuW58dm 0WF4ts3US84hr4FlRJ4xZV0ug6zLZZd1OZqsy7FlXe7qtkd0KWC5TcxLziGvtSdqtDxTo+UzaLR8 do2Wp2m0PFuj5eU61Qq1v9m8sfClHN5V5TwyDSOvwssiBiY1P7S6LFzTNIOOosQ9AbcJfSk4yyA8 Udvlmdoun0Hb5bNruzxN2+XZ2i5/j02TRLnNyktWVt4TFVqeqdDyGRRaPrtCy9MUWp6t0PJXN02i P8fkNtMuWZn2nii46pmCqz6D4KrPLrjqaYKrni246q9unZRRpSATSXTWquGeqKnqmZqqPoOmqs+u qeppmqqeranqN5qqThmj5qKq4dPDEeRVX6ePHAFAXXdt3RXF3grRCWF9ISe7B9fGa+7844d/7h9/ CZf05t9HmOSfwm2V7Rqu7BovhQpd9DkYvXv/cpxAZm9E027H1BUPXfP8HJtktJt+iz2xbzOvRaza LzaAKERd9evzECYA3WwbdPyLkWODDtceqNT+evQfp0Y5vxUurtlsPn6CiOaf/PHx5+b5qz9Zn975 Y5yj7xhXg3HbdJ+HHjgZn96JxHb1jnU/69bRGVYdOr4ZL2ATKUaIShQPj+HjH8fwhWAUOmSESfwh QPUbxwWodrgaiHWnC4Hf4zUUO5Z2bIWPLvTrMbil707Ws/diSzQ7FKnFHCFQcUbM7fsBSW2cHoD0 1KRv86aM7PQ7fVj70RvONp/PRqXehNrRqg20OVtMDtDt9HSn8W4TbeZpooTbFG3d6rrccRtX7rlN +MvgNvFGVyFgamC508DB9OHnt+blCfp4/Dn2rdlpKR/uY2YwfV7tfTzSMJD++fP7yZWnX4fr2vEE 9FNiNOM8Jc4I8/DW+c7qar5QJZ77XElV1irMp+er54azUbzqmkr71hTu3F3GyaJ3neiM0a0Ejfe/ /Mu/HPz7++v7IUaFODDp31+/HBrnQtt8HFaPDHsIn8/52DAl+bFxU5BVe9/7zyD7zyD7zyD7zyD7 uwTZGyd9CUkLa/Ofnri+oWeub+gzrG/os69v6GnrG3r2+oZ+vTFqZ52SjdlO+tp4YGRZJQ5xa4UL FmYTGGIkcUUb1cjpWeAJcnllM7tb2ZkQwjTUzHmw7gqCRmG0o7MT7FnsBIyM7AREdBwczFhx8IQw P/5UVpXWLeJcrsGidNfO5Zo+divhEqKYhro8erhTglhhtGMSjilWAIzchMOLFUYzNuH29pM5J66M FT1mL7TJ4FZmJWQwNXVN9HBLBHnCaMdkFlOeABi5mYWXJ4xmbGZh5QmTsdIunjC5foauv+P0sVt5 lpDB1NSV0fEGBUHSMNrxeCaYkgbAyMwzgZc0jGZcnomr284wyodbGUyUODA4QxAojHZMzjAFCoCR mzN4gcJoxubMVYECgzNbUYsii1qGiyXIFEY7JmeYMgXAyM0ZvExhNGNzZr2RjHPW9H1i4hg6zLq6 6LYTx/Evi8Vd01+k141S5d6gaoJcTxxPdreycyuiUeRtjmKzlARBxGjHY2fJFEQARmZ2lnhBxGjG ZWd5j7OkhNrKZxR5v6LhMgnKh9GOyRam8gEwcrMFr3wYzdhsuctZUmqrk1HkvYeGyySkTEc7JluY KVPAyM0WfMp0NGOz5eqWMCS2JBLs5A2E4mVKwt4vox2PLZK59wtgZGYLpZQzmHHZsi7lZGJLIuFN 3ilouEzC3i+jHZMtzL1fACM3W/B7v4xmbLbc5QwplchWk3cKGi6TsIHLaMdkC3MDF8DIzRb8Bi6j GZstdzlDSm0z0EpzRrmKsB3LaMdji2JuxwIYmdmi8NuxjGZctqi7nCGltnlkpTmjXEXYUmW0Y7KF uaUKYORmC35LldGMzZa7nCGlEhlkxRnlKsLOKKMdky3MnVEAIzdb8DujjGZstuydIcVjSyJ3rDij XNW7jsSWYV8UBlt6XxYPj8OnR8Px56gFdGbHIlzpBWrsMaPbChJHOBIxWFuZnBGyEyORtpWcAa1W NGJoxSOGVlhiaJWVGJom7dFsaY++yzlPOpGhldRFscNl1kRi1Exi1Ghi1HmJQZPgaLYER1890olE jEQytqSufB0u0xOJ4ZnE8Ghi+LzEoCloNFtBo68qaEjE2OZdVUldzRov0xB1MIapgzEZdDBGEqmy f00Urhi2Csbc5fAlvc2jqY4z7DCOFkSM4wUR47BBxLisQcTQZAuGLVswd9xdQScSZx1n8FGVNHpU JY8eVYmlR1VmpUdF0w1UbN1AtT65xgnXzLYrLtu46KUI/7ZnwjipquFdqWZb4bmoDIjvFhazu4JO ZEhazvOo6ogk6pgk6tAk6vKSiFYgrtgF4uouBWKdmPMajpDOGhoxrOERwxosMazJSgxLq+5ZdnXP bk52GBfuQXTx0g0HN4R/zTm6+L4bYk74V52jSyV7EeNL/Bezm6tJTJANR/BmWyKLWiaLWjSL2rws olX9LLvqZ9fHNjgxXxjqypjajs+oprv+jCrU+IwqSgyJtpNprTiT6bqgkagueCSqCyyJ6iIriWpa MbBmFwPr9dkMfe/8fNfDpoobXMZ/ZyeNisI3enjbN/MtzJs67sNa9A1qqGO2U2+tOEOdWhNppJk0 0mga6bw0olUJa3aVsF4fvPD7x6KtEkorzpqUuiGSqGGSqEGTqMlLIlrxsGYXD+v1sQpSednPtodW 3kbCxH/11WAkVafE8OlO9hgiJZI7PWd8XROLjTWz2Fiji4113mJjTSs21uxiY70+N6HsrZkdZxV+ rWNsiP/ON8xYvT3t49vH4lx8WzkUkRJpoJ4zxG6IxcmGWZxs0MXJJm9xsqEVJxt2cbJZ7zvw+z/W ErIqzxHhNcRCZsMsZDboQmaTt5DZ0AqZDbuQ2dylkGkS+UHP0ds1xEJmwyxkNuhCZpO3kNnQCpkN u5DZ3KWQaRJiKceR1rWSRoxW8ojRSiwxWmrZMk2Mlla1bNlVy3Z9HMHsmTN8bNg4t2xVeutdtbvX 7mhy+kp1K6W2KWdtOUPi1hIpZZmUsmhK2byUoi3tb9lL+9u9pf0zdvQ1bkMbMLiRRNU246wtZzjc EovmLbNo3qKL5m3eonlLK5q37KJ5u17rX7Wunc2rqnYYDhfh7X5WF63aftgjMP47O4quartmfLtr WsQDrkpknSvOkLgjltc7Znm9Q5fXu7zl9Y5WXu/Y5fVuXV4PTyavZ0QqY7E6UCP828yIFJqlH952 Zr6dZe9kGw+sDv9qDJESeeeKM4TuKiKRKiaRKjSRqrxEoq3Y79gr9rv1WQWmdPWMSCa4WGRM+HfY APXj9Hanx7fb2Rw9GGszvO20whApkXs2nCF3R9RqdEytRofWanR5tRodTavRsbUa3Xr/U12ValYI C7+amNWJ/85yz+HXQSYW/q36M5FCGOn74W3RY84krLa5Z204C12coBHJCR6RnMASyYmsRHK0df6O vc7frbdsVqGJZ2dcKmFVPOYy/jvbWzv8avT4tpnJO4JxaYa3qxL1aNvmnrXmKFQdUSTkmCIhhxYJ ubwiIUcTCTm2SMitRUJhjFPP9mkPvw7PsPjv7Eje8Gtfjm/3syKGbMJkLb4d/kUNtrf5Z605WlZH 1Ak5pk7IoXVCLq9OyNF0Qo6tE3JrnVDZVdW8GtZVg2wj/juTP4dfW3i7nZ9q2ZnaDW8bHJG2+WrD 2gXJE7VCnqkV8mitkM+rFfI0rZBna4X8WitU2n5+RGr4NQakIvyr+hmRbG/E+LaZE8n6th/e9i1G LVRt89uatUGSJ6qFPFMt5NFqIZ9XLeRpaiHPVgv5tVpItJWvZ6qzdhwjxX9nY6Twa9OObzfNmUjB WNbD25W0GCIlstqsvZM8UTHkmYohj1YM+byKIU9TDHm2YsjfZbsBm8hUs7ZJ8kQFkGcqgDxaAeTz KoA8TQHk2Qogf3W7AfryLpvIP5MPXI4X2xN1PT1T19OjdT19Xl1PT9P19GxdT3910wEGPRJZZfLZ ysPFEhU7PVOx06MVO31exU5PU+z0bMVOv1LsBDLU8wU6Yfgh4ojW1lLMVPE2hIY4bLFCx3/PI5TI JTkYqCr8jEnk2ERqmXzI8nBvRJlPz5T59GiZT59X5tPTZD49W+bT78l85sKMHll6Hw1uJVEirUw+ tC1Yu4ImCYp2HBIFeySJokVGEkU4AomCGZNEgHCnJ9Y2WWw4uyi5gibviXY8emDlPdEiKz1I8p5g xqaHXddBK6nbprSbYNNLKSvdbM4ct0WnKlPEg95nC7rAxlRN3ddmJxydIJdXNrO7lYjbZLPh7Nrk CppEKNrxiIiVCEWLrEQkSYSCGZuIa4mQzBqpEklkzu5NTtCkP9GORRCBlf5Ei5wEESTpTzDjEkRc PZGDQY9tathw9nBygiboiXY8emAFPdEiKz1Igp5gxqbH1SM4GPTYJnwN59gyJ2gynWjHowdWphMt stKDJNMJZmx63GVLlXqb8DWcs8lcSZPdRDsWMUqs7CZa5CRGSZLdBDMuMcq7HK9Rb1O9hnMsmCtp MppoxyMGVkYTLbISgySjCWZsYtzlJI16m+Q1nBPA4oZiNGLwZDHBHk2MrLKYCEciBlcWAwjZibHN 1xrOtm1O0mQu0Y5FDImVuUSLnMSQJJlLMOMSQ97lfIx6m4M1LWfwKWmylWjHIwZWthItshKDJFsJ ZmxiXD0Kgz43qRM5WM6ufk7SxCjRjkcPrBglWmSlB0mMEszY9NhsX1MZabScCZvCO0boviiappTW zLRNpjRm+LSUtbSDqm5S7oa3ZSmr8L86/oQhVSKfytnkz0makCXa8UiFFbJEi6ykIglZghmbVOut bLLmU+tEPpVzxJdTNClLtGMRRGGlLNEiJ0EUScoSzLgEUXc5P6NOZFI5p3k5RROxRDseMbAilmiR lRgkEUswYxPjLtvO1IkcKufgLqdoepRoxyMGVo8SLbISg6RHCWZsYtxj25mySORQOWd0OU3UmGim xkSjNSY6r8ZE0zQmmq0x0ffTmJRFIpPK2QzPaaLGRDM1JhqtMdF5NSaapjHRbI2J3ttCJgc9EvlU zjZ3ThOVH5qp/NBo5YfOq/zQNOWHZis/9P1OVCmLRFaVs9mdM0Tdh2HqPgxa92Hy6j4MTfdh2LoP s9nyJXRrparZ+kGjjTJ1iAWyqItmvs1CFd41ZvhbBxswTFt2xHddIIiVpRQlilSJjCxnozxniGoR w1SLGLRaxORVixiaWsSw1SJmvf1LoIk2dr5HqxHhnS5QTVYy/jQjlTS1UfEvkVimmpHKhf/1JtIw EhGxOrUsEnlcxxkGG6LGxDA1JgatMTF5NSaGpjExbI2JuYfGJBBsSwzOmXGuImpMKqbGpEJrTKq8 GpOKpjGp2BqT6h4akxB/EsTgiJ4rosakYmpMKrTGpMqrMaloGpOKrTGprmpMGEPfRCa25UieK6LS pGIqTSq00qTKqzSpaEqTiq00qe6hNCmLbSa2YmViLVFpYplKE4tWmti8ShNLU5pYttLErjdUCfMf Z6SenXkSfzOtiYNXo/tqtquq6SodIm5fmThvKme7z50/H/B8hVhuWoptFrdiZXEtUaVimSoVi1ap 2LwqFUtTqVi2SsVeValQoo3Y5m8r1hpBS9SnWKY+xaL1KTavPsXS9CmWrU+xa31KU5aVq8oLBIGP hGlymA6XBnH6TSm2Gd2KtZbPErUnlqk9sWjtic2rPbE07Ylla0/s1U1USJFkm8utWGv4aqLmpGZq Tmq05qTOqzmpaZqTmq05qdfHIjVlUYhSzJVuTZjFGlGWhYzbUc52g9Nl+M9GItSlKmYHtlUqDGTC VKpSRZxPoYYt21RuxVr4VxPlKjVTrlKj5Sp1XrlKTZOr1Gy5Sn1VrkKfQ4ttUrZiLfyriaKVmila qdGilTqvaKWmiVZqtmilvotoRSSSspyzHl1DFK00TNFKgxatNHlFKw1NtNKwRSvNHUUrYpuarSxn DNsQRSsNU7TSoEUrTV7RSkMTrTRs0UpzR9GK2KZmq4ozkm2IopWGKVpp0KKVJq9opaGJVhq2aKW5 KlohESORmq04w9GWKFdpmXKVFi1XafPKVVqaXKVly1XavW1Kxg8Ne7AVukgfwCcK+G93+7bR9PTF J4MbCVYm0rQVZ0DbEqUrLVO60qKlK21e6UpLk660bOlKe3WjE0rkKRNpWtYywpYoP2mZ8pMWLT9p 88pPWpr8pGXLT9q7yE/KRDqWtRSwI8pPOqb8pEPLT7q88pOOJj/p2PKTbn2yUFl3ohNqnnUrShvP sJdlaD1zfjR9E/EEBlEI3RRCaOHPz6pO1EJUhY/JzHJ6ZN3IqUQml7V6sCMqVzqmcqVDK1e6vMqV jqZc6djKle6OypUykZRlrSHsiMqVjqlc6dDKlS6vcqWjKVc6tnKlWx8dFBW0pjSzvbDLwtRGGlu5 UYhwJs4gntVdWUgZZbkzfcIAEXXcwaQ3EsOpbSZ34CeZU44oenFM0YtDi15cXtGLo4leHFv04jan CIXpkDPnMvQ3oWyhhulQWfRFO2dU3Fe9reJfpZufJSzgw04J3CNsmwCuWs542RH1Lo6pd3FovYvL q3dxNL2LY+td3PowITmsRi2aM6GsLuui7wpdFT08zaatj+tCDX+0w79nQimtA0xbmqL13koUpxJZ 45Yz1HZEqYxjSmUcWirj8kplHE0q49hSGbeWyogoOlfFbDttYaUN77YyvBv3Fp2NtYcd98Mfh5g0 TwuJ0sdAJawf/orhVCLV3HCG2o6opnFMNY1Dq2lcXjWNo6lpHFtN47BHEp3ShTJM2lTp7DKbuPyO 2cdupVQiSd1whueeqMPxTB2OR+twfF4djqfpcDxbh+Pvd4xRKRMp5pojL/dESY1nSmo8WlLj80pq PE1S49mSGo+V1HwLY5dz3eJGniQyzjVHMe6J2hrP1NZ4tLbG59XWeJq2xrO1Nf6qtoYRRhJ5Z8vR jfdEhU3PVNj0aIVNn1dh09MUNj1bYdNfVdiQiLFNHlvJGdH2RG1Nz9TW9GhtTZ9XW9PTtDU9W1vT 31FbI7fJYys5o9OeqK3pmdqaHq2t6fNqa3qatqZna2v6O24IIxN54JIxOg0vJHpEOw49wguSHtEi Iz0iHIEe4YVJD0BY1Ba8NyMfbpkY+0IZrUSzkdmsv+z0sVu5tc0J25IxovUFTVwT7XjcwoprokVW bpHENcGMza37nSJUym161wrGiNYXNIlNtOPRAyuxiRZZ6UGS2AQzNj3uIrGR2xytFQy1uBc0iU20 YxFDYCU20SInMQRJYhPMuMQQV3d4YcSNbb7VFgy1uBc0tUy049EDq5aJFlnpQVLLBDM2Pe6ollHb fKstGJpxL2hqmWjHowdWLRMtstKDpJYJZmx6rNUyWgobenGjHe+bqrauXRSix780Rd1W4Y/bKlBv W9H34S87Z/BOkOuy0MnuVh5u87m25Yx+S5rCJtqxeFhiFTbRIicPS5LCJphxeViuFTa+VNr3yp+J WGlfN03V+mFD0jkRw1+MrWVduUCb3rkzEW1Vdk2nvNBlYClKtqW2WWDbcMbMJU1mE+14pMLKbKJF VlKRZDbBjE2qu2wroxJZ4IYzZi5pWploxyMGVisTLbISg6SVCWZsYqy1Mm1TdK0UN6dxiiJaWF1f TuOcPrZ++I0XolRvNg8/p2vZl/2tZEzknGvOCL2kiWyiHY+MWJFNtMhKRpLIJpixyYgV2WBiVSLn XHNG6JImmIl2LHpIrGAmWuSkhyQJZoIZlx7yjoIZlUgb1wxpupc0wUy049EDK5iJFlnpQRLMBDM2 Pe64B41KpI0VQ2nuJU0nE+149MDqZKJFVnqQdDLBjE2Pu+xBoxJpY8WQi3tFU8hEOxYxFFYhEy1y EkORFDLBjEsMdReFjEokjD1nOKpoCploxyMGViETLbISg6SQCWZsYlxVyFCIoROpYs8ZiCqaNiba 8YiB1cZEi6zEIGljghmbGHfZd0YncreeMwTVRFWMZqpiNFoVo/OqYjRNFaPZqhi9t+8MjxiJ/Kvj DD41UdKimZIWjZa06LySFk2TtGi2pEXfZb8Ynci/Os7gUxPFLJopZtFoMYvOK2bRNDGLZotZ9F3E LDqRC+0Y8mxviGIWwxSzGLSYxeQVsxiamMWwxSzmLscV6UQWtGMIs70hylgMU8Zi0DIWk1fGYmgy FsOWsZirMhYSMRL5z5YjyTZEAYthClgMWsBi8gpYDE3AYtgCFnOPg4pk4uS7VtHzGKrwpOL/YMcg RrTHEWOwyEeMAQ5PjGjGI8aEkJsYicOGWkkffKqiJ1VMBjsWMXpkxWSwyEmMnlIxiWZcYvRXKyYk YmzzGG1JH3yGyyTVSgY7HjGQtZLBIisxKLWSaMYmxj1qJTJxqFBb0gefShSkWslgxyFGsEcSI1pk JEaEIxAjmDGJAQjZibHNY/QtPfMZLpNUKxnseMRA1koGi6zEoNRKohmbGPeolcjEyUCtpWc+w2WS aiWDHY8YyFrJYJGVGJRaSTRjE+MetRKZOBOotYzBpxCkWslgxyKGQNZKBoucxBCUWkk04xJD3KNW IhNnArUVY/ApBKlWMtjxiIGslQwWWYlBqZVEMzYxVrWSb7YvTYmSJAcL3V+XJMPHbqXWVgPWVpzh qyBVWwY7HrWQ1ZbBIiu1KNWWaMam1tVqC1kiKBPnCbWGnkFVoiTVXAY7Fj1KZM1lsMhJj5JSc4lm XHqU91tALBOnCrWGvjIvXCyp8jLY8eiBrLwMFlnpQam8RDM2Pe63gFgmzgRqNX2NXbhYUv1lsOPR A1l/GSyy0oNSf4lmbHpcrb8w6LHNqfaeM6yVpHW9gx2LHhK5rnewyEkPSVnXG8249JDFHemxzaz2 njM0lbQiXbTj0QNbpIsWWelBKtIFMzY99op044eGc+cqaZJH1u0eVDca3EqiRBbWcQawkrSad7Dj kQi5mnewyEoiymreaMYm0Xo1b5k1ymyzsb3jDGElaYXtYMcjCHKF7WCRlSCUFbbRjE2Q+62wlYnT ffqOM4RVpBW2gx2LHgq5wnawyEkPRVlhG8249FD3W2ErE4f19B19s4hwsTS9SLTj0QOrF4kWWelB 0osEMzY97rfCVibO3elbhs5MKJpqJNrx6IFVjUSLrPQgqUaCGZse99uJXiaO0Olb+qo5JTRRO6KZ 2hGN1o7ovNoRTdOOaLZ2RF/VjjDoscmutgVj199wsUQFiWYqSDRaQaLzKkg0TUGi2QoSfb/96OX2 OJy2YOz6Gy6WqCPRTB2JRutIdF4diabpSDRbR6Lvtx+93J6CE+jBGZoaoprEMNUkBq0mMXnVJIam JjFsNYlZ70cvVCukwKgGokXpr6oGpo/dyq1NarYtGBsHhzslClIMU5Bi0IIUk1eQYmiCFMMWpJi1 IEVtk7OlanQ6OftN7aRnJ5PTV6pbKbVJ1AZKccbChihEMUwhikELUUxeIYqhCVEMW4hi7ihE2R6+ 0xaMU7aVqIhClIopRKnQQpQqrxCloglRKrYQpbqjECVx+E7fczTVFVGIUjGFKBVaiFLlFaJUNCFK xRaiVHcUoiTOz+l7jrK6IgpRKqYQpUILUaq8QpSKJkSp2EKU6i4Lgbcn57SF5gxkLVGCYpkSFIuW oNi8EhRLk6BYtgTFrreWr5q6K1qMsjpYtM5dVVbDx26l1ia5G6jFGdBaonzFMuUrFi1fsXnlK5Ym X7Fs+Yq9usac8UhKJHc1Z0BricIUyxSmWLQwxeYVpliaMMWyhSl2LUypXF0Z1JqOaCGvr+mAj91I re25PW3B2Nc33ChR0mKZkhaLlrTYvJIWS5O0WLakxd5R0rI9TifQgzMYromSlpopaanRkpY6r6Sl pklaarakpVbXk3dFVdUal7ybTNDJu+1hOm3BODk73CBRBlMzZTA1WgZT55XB1DQZTM2WwdR3lMFs j9QJ9OAouWuiDKZmymBqtAymziuDqWkymJotg6nvKIPZHnLTFoyTs5VoiDKYhimDadAymCavDKah yWAatgymuaMMZnvITVvUnOFsQ5TBNEwZTIOWwTR5ZTANTQbTsGUwzV02UtkebxOIwRnINkQBTMMU wDRoAUyTVwDT0AQwDVsA09xlI5XtwTZtYTnD0ZYofWmZ0pcWLX1p80pfWpr0pWVLX9qrG6kwHiiJ zKzlDEdbonqlZapXWrR6pc2rXmlp6pWWrV5pr249z6BHIjNbcYajLVGJ0jKVKC1aidLmVaK0NCVK y1aitN06M1u19v9n7216Jbexdc25f0UO7/W5A1EUv0YF9Mcd9aCBRo8T/CwYzrLdmVl1ss6vb1LS 3qEMMbz3Wi9lxwWcBbgyY8errYh4RDEWH5KeNiZUE+YdY0Lb096J1nmbnIoWYnRHpsUSQYslki2W ONZiiTyLJcIWS7xiCXt53iYnTBpxuSPTX4mgvxLJ/koc669Enr8SYX8lXuivnDfLqXggLndk+isR 9Fci2V+JY/2VyPNXIuyvxEv8lfNmOWFKSFc2Mf2VBPorieyvpLH+SuL5Kwn2V9KbS6iwwOjUVCPS iU1M+ySB9kki2ydprH2SePZJgu2TdMkOB+fNcioYSBc0Mb2TBHonieydpLHeSeJ5Jwn2TpK/BIxO NTUgXdDEtEYSaI0ksjWSxlojiWeNJNgaSW9aIywwOtXUgHQ+M9MXyaAvksm+SB7ri2SeL5JhXyS/ uQQKA4zlvDZOkAXoY0jJG5hrOQSMmieC0RIDwWiHY4BRYyAY+xEOhbKU2+085ZtP9PLI4SdfHv4k 9H9Cgep8G7ITMNorJW9Qr+UwqKiDei0xFCrWoF6NwVClZ4Oqs1SCyUhLpXh115aDoFLUumtLjIRK sequNYZCpcTTQXWu2ZoMfJOqPXAmVFjNtubJUA2t2bbDsaBCa7b7EUb3izqrI5gEfJOSiletbTkM DGq1tiWGgsGq1tYYDMYV1dqls8aBScA3Kal51dqWg8DQ1GptS4wEQ7OqtTWGgqGvqNYundUNIrCD dT1NXrW25TAwqNXalhgKBqtaW2MwGFdUa5fzugYhAjtY19PkVWtbDgODWq1tiaFgsKq1NQaDcUW1 djmvaFDBQDqfmletbTkMDGq1tiWGgsGq1tYYDMYV1dqls6JB9Ejn0/CqtS0HgWGo1dqWGAmGYVVr awwFw1xSre2sRxA90vk0vDl6LYeBQZ2j1xJDwWDN0asxGIy7OXrfrBdZvWPCOQWTs/0aHbAiijS8 uXoth2FCnavXEkMxYc3VqzEYkys2Ol86qwpEB6xnIi1vll7LQWBY6iy9lhgJhmXN0qsxFAx7xUbn S2c9geiAlUykZQ4DWnAY0JKHAe3YYUDLGwa08DCgfTQ/b3tSWxrAB+9J+zTtgfdCdK6ZxoJ0Wy1z 2M+Cw36WPOxnxw77Wd6wn4WH/eyjuXw3JoyrhJM2+9oC74XoXF+NBeniOt68v5aDIHLUeX8tMRIi x5r3V2MoRO5+yWvls/FbX/aOpnoX9Ut/gZP9gMtS/6+FH+D1coT7E3nN/ZBCO+eYP37Jn8rH/6yn 9Fv+vB2m+6P2AYXT64o+S7HUz+fnGvqc8y9fXw5yeKBF44Nomn6I/tOn9pZsuZd/tU/icebwNsa7 96+9AWISzpTDl4bvDqD8+Q3dfqLl9oau515JCv/+mr+8vim3h+rJ+dOUqddDtPjf89eP//Kf/plf 06+P1LA8o7mFzRoOPv68fgKv4ddH2pWV3IN0OXys27Vw94FuD7YTODcK6yGMqBdnffqXr/UX7qH6 gexX6vkH9VAlPThUWM9mb9ZeT2T/dzuH6UHSbu/Cl1g/16/1sszxNX14rL0T/gEiThyPUFE8gHl+ vB5pOV/024HUy1v62/GtbHTmB5+hy9vVcMv8fAvN6tyqrqlQsbklXi6A+OCTjop+2bTM4bKRnctm Ci4oNz+4bNL86LKpP1kvm/ZC75qAlzdYPniDa/SHf/3mf/lp/4y3v7fPVj94p3J9HYfAy/OXR09v GFboP/38+fVSfvnnel4PrgT6XWKNQXeJ1yMcm7eYo1XmsAyWLbVPa+Qyt9XJrBC3j2tuQxl5id6o HPSUbh+XTnIqKYqotQrS6fXj+vHHHz/kz59//fyhtQqtD1I+//qPDz6l+t58+XB3y7Af6vNH3jb0 zL5tvKuRXR793r8a2b8a2b8a2b8a2T+kkX3n97uOJpGR2rTjLdzRctj3O+rCHS0x9Psda+GOGoO/ 31EX7nhtEvY/FEOzs3JYzEjJ2vFW8mg5jBfqSh4tMZQX1koeNQbz8uaeMqySdceeyEjJ2jN9cA/6 4J7sg/uxPrjn+eAe9sH9/d1pXorKLtpe2VEG0S8UiTyJUMS0/XlYhdzyr6fxfeq9zHXEjIRMbPFM XdyDurgn6+J+rC7uebq4h3Vxf6eLH3quA0beO8uSxQQsZyg9Uxv3oDbuydq4H6uNe5427mFt3F+i jXcWJIsRWM6wrRHCAiOA2ngga+NhrDYeeNp4gLXx8KY2zm83OouKJYn0cgNTHg+gPB7I8ngYK48H njweYHk8XLfRzNJZWqx+A0daD6ZCHkCFPJAV8jBWIQ88hTzACnl4UyEH8DiLF2lGuqWBKZIHUCQP ZJE8jBXJA08kD7BIHi4RyTtLi6UZ6Y5GpkgeQZE8kkXyOFYkjzyRPMIieXxTJAfajXMtNgmkUxqZ OnkEdfJI1snjWJ088nTyCOvk8botX5bOMmNJIDMaI1Mjj6BGHskaeRyrkUeeRh5hjTxet+XL0lls LE3IvMbElMkTKJMnskyexsrkiSeTJ1gmT9dt+bJ0lhxLEzK7MTGV8gQq5YmslKexSnniKeUJVsrT m1u+dAf67CQnobw4VeO//x2Hp70XqXN1NRmkO5uYgnkCBfNEFszTWME88QTzBAvm6c3NYoAW51xj TRrpzmamOp5BdTyT1fE8Vh3PPHU8w+p4vm7LmEV3aqwa6c5mpnmSQfMkk82TPNY8yTzzJMPmSX7T PGGB0amuKqQjm5mKSQYVk0xWTPJYxSTzFJMMKyb5TcUEaDc61VWFdGQLUzQpoGhSyKJJGSuaFJ5o UmDRpNyLJmqOtW8bezOSfK5fm6f7GTX1J04VXVu+c29XqJikmfTDKUr7Ie+7v6+5+0yeZKmZNgH3 nMnKyVA7NY3VT7++ONntQ28jQneBlhFimZ3376W9UzJekEmcham4FFBxKWTFpYxVXApPcSmw4lKu 28Vm0Z2S8YJMzyxMwaWAgkshCy5lrOBSeIJLgQWXcie4fMvGinKcnvnW1/uStVrKm1/vX5/2XrY6 9WYJqOF+4o1WtRzCVs0T2WqJgWy1wzHYqjGQrf0IwzvonUqzBzrofuKNU7UcBgZ1nKolhoLBGqeq MRiMN8epWGB0aswe6Jr7iTdC1XIYGNQRqpYYCgZrhKrGYDAuWehIdyrFDujFesEbm2o5CAxBHZtq iZFgCNbYVI2hYIhLFjrSnRqxA/qvXvBGpVoOA4M6KtUSQ8FgjUrVGAzGm6NSHDBMpzpskc6n4I0t tRwGBnVsqSWGgsEaW6oxGIw3x5ZYYHSqwxZQs/3MG1VqOQiMmTqq1BIjwZhZo0o1hoIxvzmqxAKj Uxe2gJTtZ954UsthYFDHk1piKBis8aQag8G4ZDzJdEqoBtCx/cwbT2o5DAzqeFJLDAWDNZ5UYzAY l0xZNp3iaUY6n5I3ktRyEBiSOpLUEiPB4CyoscZQMO4X1BgERqfymZDOp+QNurQcBgZ10KUlhoLB GnSpMRiMS7ahMp3KZ0I6n5I33NJyGBjU4ZaWGAoGa7ilxmAwLplPbDqVz4R0PhfefOKWg8BYqPOJ W2IkGAtrPnGNoWAsl2xDZTqVz4h0PhfeTOKWw8CgziRuiaFgsGYS1xgMxiXbUJlO5TMC8rRfeHOI Ww4DgzqHuCWGgsGaQ1xjMBiXbENlO5XPAGjTbTlmHhjY7OGaJ4MxdPZwOxwLDHT28H6E4WB0Kp8B EKa9YvoYCvQxFNnHUGN9DMXzMRTsY6hLfAx7rnzWNxhoMRTTx1Cgj6HIPoYa62Mono+hYB9DXeJj 2HPlMwuk86mYPoYCfQxF9jHUWB9D8XwMBfsY6hIfw54rn1kgnU/N9DE06GNoso+hx/oYmudjaNjH 0Jf4GPZc+cwT0vnUTB9Dgz6GJvsYeqyPoXk+hoZ9DH2Jj2HPlc88IZ1PzfQxNOhjaLKPocf6GJrn Y2jYx9CX+Bi2U/ksiAxsmD6GAX0MQ/YxzFgfw/B8DAP7GOYSH8N2Kp8FkYEN08cwoI9hyD6GGetj GJ6PYWAfw1ziY9hO5TMjMrBh+hgG9DEM2ccwY30Mw/MxDOxjmEt8DHeufGaFdD4t08ewoI9hyT6G HetjWJ6PYWEfw17iY7hz5TMrpPNpmT6GBX0MS/Yx7Fgfw/J8DAv7GPYSH8N1Kp8L0vm0TB/Dgj6G JfsYdqyPYXk+hoV9DHuJj+E6lc8F6Xw6po/hQB/DkX0MN9bHcDwfw8E+hrvEx3CdyqdEOp+O6WM4 0MdwZB/DjfUxHM/HcLCP4S7xMVyn8ikRGdgxfQwH+hiO7GO4sT6G4/kYDvYx3CU+hutUPmdEBnZM H8OBPoYj+xhurI/heD6Gg30Md4mP4c6VzzwjMrBn+hge9DE82cfwY30Mz/MxPOxj+Et8DHeufGaH dD4908fwoI/hyT6GH+tjeJ6P4WEfw1/iY7hz5TNbpPPpmT6GB30MT/Yx/Fgfw/N8DA/7GP4KH0NN ncqnRTqfgeljBNDHCGQfI4z1MQLPxwiwjxGu8DHU1Kl8GqTzGZg+RgB9jED2McJYHyPwfIwA+xjh Ch9DTZ3Kp0E6n4HpYwTQxwhkHyOM9TECz8cIsI8RrvAx1NSpfBpEBo5MHyOCPkYk+xhxrI8ReT5G hH2MeIWPoaZO5VMjMnBk+hgR9DEi2ceIY32MyPMxIuxjxCt8DDV1Kp8akYEj08eIoI8RyT5GHOtj RJ6PEWEfI17hY6ipU/mMSOczMX2MBPoYiexjpLE+RuL5GAn2MdIVPoaaOpXPiHQ+E9PHSKCPkcg+ RhrrYySej5FgHyNd4WOoqVP5jEjnMzF9jAT6GInsY6SxPkbi+RgJ9jHSFT6GmjqVz4B0PjPTx8ig j5HJPkYe62Nkno+RYR8jX+FjKNGpfAak85mZPkYGfYxM9jHyWB8j83yMDPsY+QofQ4lO5dMjMnBm +hgZ9DEy2cfIY32MzPMxMuxj5Ct8DCU6lU+PyMCZ6WNk0MfIZB8jj/UxMs/HyLCPka/wMZToVD4d IgMXpo9RQB+jkH2MMtbHKDwfo8A+RrnCx1DiXPksE9L5LEwfo4A+RiH7GGWsj1F4PkaBfYxyhY+h xLnyWSak81mYPkYBfYxC9jHKWB+j8HyMAvsY5RIfQ3QqnwXofIaJ52O0HAJGzRPBaImBYLTDMcCo MRCM/QjDwehUPgvQ+QwTz8doOQwMqo/REkPBYPkYNQaDcYmPITqVzwx0PsPE8zFaDgOD6mO0xFAw WD5GjcFgXOJjiE7lMwMycBA8H6PlIDAE1cdoiZFgCJaPUWMoGOISH2PuVD4TIAMHwfMxWg4Dg+pj tMRQMFg+Ro3BYFziY8ydymcCZOAgeD5Gy2FgUH2MlhgKBsvHqDEYjEt8jPlc+SwL0vmceT5Gy0Fg zFQfoyVGgjGzfIwaQ8GYL/Ex5nPls0ik8znzfIyWw8Cg+hgtMRQMlo9RYzAYl/gYc6fyKZHO58zz MVoOA4PqY7TEUDBYPkaNwWBc4mPMncrnjHQ+Jc/HaDkIDEn1MVpiJBiS5WPUGAqGvMTHmO8rn4sS GgAjMn2MCPoYkexjxLE+RuT5GBH2MeI1PsZpkfnF6La3+4f/9r//3//vh9n+9x9+TuFj/uVr/ryd 7es/63nK80uNPkuR6kf0r9/8Lz/FLbP9vX2o+gzFmshq+uEQeHn+8ujpYfoh1Y/q08+f85ct8vrP 9bzOzG458ke3xqCP7vUIt4/uWxPUrTLL62f4zRZbpJHL7JYUrRCvH+q3uYkqeYneqBz0lF4/5W86 1cYiRRG1VkE6TfnY73sQ9WPnb9AdfQlrT4DYHqy5PLHbgy2/1N+7Pvvjl99++uXjl7B/sncPtmMs 6cExeC3E43NicFZjIGf7EYY3EfedisoKf8/ueppx4rESJ4yVmodZidxuxuNz4rASsX7G6xEObdIs p2kpk721SVIrJdoCB8Kn+n83ir7VL/ptD+apdvPm+QbVN2nlIuQkYw1NKlsKZaceSqWMPfmgvUDF pEyBlKkBlHH7LI/PiUUZ1ml5PcKBstAmrdZP7EZZrl+SVf1CZNVU5krU4c5n50VFGxZnKyHG3Thb JjtXflUl0AcpFIWz+6Hexhl7LkN7iRwBdcvxe8JrntQT3hLDesLb4VhQYQLq6xE6t7lvJd3oKYlx s9OdedaSvYq1NLk4RpV+z7Hx2PIUPPbEKDz2w1Hx2GIIHrcjDO4F6c48a8neVbzeWaVj1Nz2HB+M NU8CY0sMA2M7HBmMNQaB8XqEbruxP8kV51xJ9tCK3P35Hp2//W07gfMR3kvVuWAn2VuSt9foGQW7 PQdR5UkFuz0xkipP70hvMZQqf0XBTncmaUv2luTraTIKdnsOA4NUsNsTQ8Gg9323GAzGo4Ld7WY1 OWsfNzd5mt5qbl6O8F6qzp6jZO9nvr5GRud3z2FUkTq/e2IoVfTO7xaDqbpi9pXuzPCeDXcuhbQp WYYyv+f4YKx5EhhbYhgY2+HIYKwxCIzXIwwH4+w5zoY7l2LllzEtb89hLQZpWt6eGNpi0KflbTG4 xXg0Le/YafUJ7fauR3gnVXfTw1uhUbMlyfYaA2NO356DqAqkOX17YiRVgT6nb4uhVIUr5vTpu+nh GxjIt+zAmNO35zAwSHP69sRQMOhz+rYYDMajOX3HxiIYtLlZj/BequYzVewVr+oXPslZbnXP8ala 8ySqtsQwqrbDkalaYxBVr0d4V3Pzzcy6mNnMlHZHdghhf9320XFWUdxzACEtTyNkTYwjZD0cnZAW wwh5OcJhqEn5orZRyxfHYppScsav/x+0wu5US4cY9lfpRjhjZtiew9oU0sywPTG0TaHPDNticJvy aGbY4T4TQgDvVNsR3kvVXT04JqEW9vfw+ho5y/zuOYgq2jK/e2IkVYxlfrcYStUly/zq+6nrKxjs 7+HtNFk3qDWHgUG8Qa2JoWBwblAtBoPxaFrZsbGICm1u1iO8lypzpkpyF0RYXyNntHvLYVTRRru3 xFCqGKPdawym6pLR7rt57/UXKcG2sdppctYI3nMQGLQ1gvfESDAYawRvMRSMh2sEf9dYFLi5KYTm xnWoQvrMnAWG9xxGFWlC254YShV9QtsWg6m6m9D2TZwbnG/L7bv4y50lCxPbt7HvQTrZXS9PeydQ 87mObNhLgK0vjyPlbDkMKJqUsyWGAsWQctYYDNQVzrqez3Vkw14CrJ0mZ2HiPQeBQVuYeE+MBIOx MPEWQ8F4uDDx8e6TMnr/Wo/wXqo6dWT2+mHra+RIOVsOo4om5WyJoVQxpJw1BlN1xSw6PXfKx+z1 w9bT5Hg1Ww4Dg+bVbImhYDC8mjUGg/FYKr81Fvl3bK33NTeZYGvNnRIze/Gx9TVy3Isth1FFcy+2 xFCqGO7FGoOpumJJZH03zX8Dg7v+g4zFFU5zs+X4YKx5EhhbYhgY2+HIYKwxCIzXI/weGC/PFjLn MC+BR4o+f+OO7H5wO29OE7LlMFJoTciWGEoKowlZYzApb+pbaTIF07dejvBeqsyZqoD0gzlLsu85 6MZEW5J9T4y8MTGWZN9i6I3p4ZLsx85KmdHuznqE91LVqTkHpBPNWc99z2FU0dyvLTGUKob7tcZg qq5Yz13PnbKxR/rBnPXc9xwGBk1O3xJDwWDI6WsMBuORnP5dYxHh5ia+v7mRU4cqdie6iY8cqrYc pprSqNoSQ1VTBlVrDFZNL5nyIMUZDIfI6ZGxS8Ceg8CIpF0C9sRIMCJ9l4AthoIRH+0ScGgsXBSo g7we4b1UzR2qELM9chzkLYdRRXOQt8RQqhgO8hqDqbLvGPt8wccKp5elHLTklyPFeqOybw6Fvj7t hxTaucX88Uv+VD7+Z/3Vv70sxtb9UfsgwmlkdV2GbKmfQ1vA7XO+W9Ftf6BF44Nomn6I/tOn9tK3 3Mu/2jv+OHNc9ezuMlzfpUk4U85Vj+0Ayp/fv+0n7fbd3r/13Csx4d9fX1eCOz5UT86fh5lfDtHi f89fP/7Lf/pnfk2/PlLDsrf0SQubNRx8/Hn9BF7Dr4+0Kyi5B+ly+Fg35u8+0O3BdgIPlrIzol6E 9elfvtZfuIfqB7Jfkecf1EOV3lpD7VBhPZu9+Xo9kf3f7RymB0m7vQtfYv1cv9bLL8fX9OGx9k74 B4g4cTxCRfEA5vnxeqTlwRp9Tr28pb8d38pGZ37wGbq8XQ23zM+30KwerD0YKja3xMsFEB980lHR L5uWOVw2snPZTMEF5eYHl02aH1029SfrZfPXoo7/Sy3q+OOPP37Inz//+vlDaxVaX6N8/vUfH3xK 9b358uHulmE/1OePvG3omX3beFcjuzz6vX81sn81sn81sn81ss+zcq6WsuOwIis2Rc68ry2HfY+j zfvaEkO/xzHmfa0x+HvcFTuCaXlna8QSloVdpa6nmThTt7YcBEaiTd3aEiPBSIypW2sMBSM9mrp1 LPqkCS0brUd4L1WqQxVSpU6ceV9bDqOKNu9rSwylijHva43BVF2xnZiW+gyGRKrUiTN1a8thYNCm bm2JoWAwpm6tMRiMR1O3vmssPNzceEJzYzpUIVXqzJn3teUgqjJt3teWGElVZsz7WmMoVfmKvci0 tGcwZqTbmzlTt7YcBgZt6taWGAoGY+rWGoPBeLQX2bGxyAva3KxHeC9VrkMVssppZi6qnMFFlTN5 UeU8dlHlzFtUOcOLKudL5m8tUwcM9kzRkkvhzLTZcnww1jwJjC0xDIztcGQw1hgExusRDqWbdUH+ x4s2uenwh4JKR9eI7Dmg7cQ5MvyWw1ChyfBbYigqDBl+jcGo3Mnw39I6/ynl2y3q5ZHDT748/Eno /4QC1Z2tUcoE9IPbS+R481sOg4rmzW+JoVAxvPk1BkN1ydSbZTmDAfWDWTemLQf1WIg3pi0xssfC uTGtMbTHcn9jGgSG6oCBdGVZt6Eth4FBXlp95G1oOxwLDHhp9Ydzso5fbwo6BXQ7wnupMh2q2LtV ZWE4C/bvOf5uVWuetFvVlhi2W9V2OPIQ5hqDhjBfjzC8uenUYzS317ueJqODsucwMEgdlD0xFAx6 B2WLwWC8ObFPegOuy/5yhPdS1anHaO500fYaOeuy7zmIKtq67HtiJFWMddm3GErVNeuyq049RnFn fK6nyZibt+cwMEhz8/bEUDDoc/O2GAzGm+uy18YCXJf95QjvpUp0qOLO+FxfI2MK1p7DqCJNwdoT Q6miT8HaYjBVl0zBUufyi2PvU7UuKseYgrXnIDBoU7D2xEgwGFOwthgKxttTsGpjAU7BejnCe6mS 5y9TUJ+ZMwVrz2FUkaZg7YmhVNGnYG0xmKr3bwNBaW46RT2o28txOvccBgbJ6dwTQ8GgO51bDAbj zbX8W2OBreX/coT3UtWpCEJ9Zo4QuucgqmhC6J4YSRVDCN1iKFUPhVCsudEdMJBuL8fp3HMYGCSn c08MBYPudG4xGIw31/KvjUXC1vJ/OcJ7qeqUitn++foaGULonsOoIgmhe2IoVXQhdIvBVF2ylr+y HTC4Cnk7TY7TuecgMGhO554YCQbD6dxiKBgPnc7vGgtsLf+XI7yXKtcxJLj++foaGULonsOoIgmh e2IoVXQhdIvBVD0UQgXU4OjphIafkI4vx+rccxgaJKtzTwxFg251bjEYjUdW57G5yA5tcNYjvJcq 0aEK6TUXxpL+ew6iqpCW9N8TI6kq9CX9txhKVXm0pD/W3HSKxQXp+HKUrD2HgUFSsvbEUDDoStYW g8F4pGQdG4si0eZmPcJ7qToXi11Bes0cn2vPYVSRzZuBPtd+OBZVsHnz9hrbrObmXCx2Gen4ctTg PYeBQTZvyljzpvDMmwKbN+U95g24pPbLEd5L1blY7DJbOJfBLBx9dMvx9dE1T9JHt8QwfXQ7HFkf XWOQPvp6hGeaxaDPhWaX2LJ6e4mcWQxbDoOKNothSwyFijGLYY3BUJWng+pcZ3aJLbrXl6g4i/9v OQgqRVv8f0uMhEoxFv9fYyhUank6qM41aq/Zy5y0l8hZ+3/LYVDR1v7fEkOhYqz9v8ZgqNzTQXUu UXvFXiKlvUTOCu9bDoOKtsL7lhgKFWOF9zUGQ5WfDSrTKW4rpKOuOavDbzkIKk1bHX5LjIRKM1aH X2MoVFo+HVSd2vaCdNQ1Z3H4LYdBRVscfksMhYqxOPwag6G6Xxz+z4fqXBf3C9JR15yVKrccBhVt pcotMRQqxkqVawyGKj0dVOeyuF/YC7PUl2g4q1xuOQgqQ1vlckuMhMowVrlcYyhUZn46qM4ldS/Y S7i0l8hZ5HLLYVDRFrncEkOhYixyucZgqMzTQXWuqHvB3vq0vUTOAplbDoOKtkDmlhgKFWOBzDUG QxWfDqpzRd07pKNuOetjbjkIKktbH3NLjITKMtbHXGMoVPZ+Sfk/H6pzRd07pKNuOWtrbjkMKtra mltiKFSMtTXXGAyVfjqoOhV1h3TULWdpzS2HQUVbWnNLDIWKsbTmGoOhCk8HVaeibpGOumM4mHsO gsqRHMw9MRIqR3cwtxgKlZueDSrbqahbpKPuOEvqbTkMKtqSeltiKFSMJfXWGAzV/Vqvfz5UnYq6 4c5BWF8i06dyoE/lyD6VG+tTOZ5P5WCfyj2dT2U7FXXDnYKwvkSmT+VAn8qRfSo31qdyPJ/KwT6V ezqfynYq6pq9b1R9iZ7pU3nQp/Jkn8qP9ak8z6fysE/ln86nsp2KekI66p7pU3nQp/Jkn8qP9ak8 z6fysE/ln86nsp2KekI66p7pU3nQp/Jkn8qP9ak8z6fysE/ln86nsp2KekQ66oHpUwXQpwpknyqM 9akCz6cKsE8Vns6nsp2KekQ66oHpUwXQpwpknyqM9akCz6cKsE8Vns6nsp2KekA66oHpUwXQpwpk nyqM9akCz6cKsE8Vns6nsp2KekAmPkSmTxVBnyqSfao41qeKPJ8qwj5VvGSRONepintk8kJkOlER dKIi2YmKY52oyHOiIuxExUs2/nWdyrZHJiBEptcUQa8pkr2mONZrijyvKcJeU7xknTd3rk4H9rKS 7TQT001KoJuUyG5SGusmJZ6blGA3KV2yd687V5gDe2XI9TSZflEC/aJE9ovSWL8o8fyiBPtF6dFS bRgY5ypxYK8MuZ4m0xFKoCOUyI5QGusIJZ4jlGBHKF2y/a47V3oDe3HHdpqZ6flk0PPJZM8nj/V8 Ms/zybDnk5/O83HnSm/g76naXiLT88mg55PJnk8e6/lknueTYc8nP53n486V3sDfj7W9RKbnk0HP J5M9nzzW88k8zyfDnk9+Os/HnSu9gb+Xa3uJTM8ng55PJns+eaznk3meT4Y9n/x0no87V3oDeyvX 9hIL0/MpoOdTyJ5PGev5FJ7nU2DPpzyb52Omc5U4RKSjXpieTwE9n0L2fMpYz6fwPJ8Cez7l2Twf M50rzCEiHfXC9HwK6PkUsudTxno+hef5FNjzKc/m+ZipU50O7I76pIPmjGdtOT5Ua54E1ZYYBtV2 ODJUawyC6vUIvWVzD/CIyU9FzaoN7ig5uxs8M6ECZaZOzTqwu+Dt5DmjXFsOw4U2yrUlhuLCGOVa YzAuj0a5vrkbLloXq4NJi/VRiJlLS6eQHdh963ruhjP0teUgWgxt6GtLjKTFMIa+1hhKi7li6MtM nUI2e8vX9TQ5Q19bDgODNvS1JYaCwRj6WmMwGFcMfZmpU4xmb/m6niZn6GvLYWDQhr62xFAwGENf awwG44qhLzN1CsrsXVvbaVrO0NeWg8CwtKGvLTESDMsY+lpjKBj2im2GzHQuCscJ6ZFazvDVlsPA oA1fbYmhYDCGr9YYDMajbYYwMM6F3TghnU/LGYLachgYtCGoLTEUDMYQ1BqDwbhipyAjOsXZgnQ+ LWcYacthYNCGkbbEUDAYw0hrDAbj0U5BGBidAmtBOp+OMxS05SAwHG0oaEuMBMMxhoLWGAqGWy4B o1MkzUjn03GGc7YcBgZtOGdLDAWDMZyzxmAw3CVgdMqhme12t9PkDMlsOQwM2pDMlhgKBmNIZo3B YORLwOhUPhPb7a6n6TnTp7ccBIanTZ/eEiPB8Izp02sMBcPLS8DoVD4T2+1up8mZAr3lMDBoU6C3 xFAwGFOg1xgMhr0EjHPlMy5I59NzpjFvOQwM2jTmLTEUDMY05jUGg5EuAeNc+YwS6XwGzlTkLQeB EWhTkbfESDACYyryGkPBCFdMRTaiU/mUSOczMNWNAKobgaxuhLHqRuCpGwFWN8IVU5GN6FQ+Z6Tz GZiSRgAljUCWNMJYSSPwJI0ASxrhiqnIZj5XPuOMdD4j08eIoI8RyT5GHOtjRJ6PEWEfI17iY8zn ymcUbF+5nSbTx4igjxHJPkYc62NEno8RYR8jXuJjzOfKZxRs57idJtPHiKCPEck+RhzrY0SejxFh HyNe4mPM58pnnBBvODF9jAT6GInsY6SxPkbi+RgJ9jHSJT7GfK58Rv5ybO00mT5GAn2MRPYx0lgf I/F8jAT7GOkSH2M+Vz4jf0m1dppMHyOBPkYi+xhprI+ReD5Ggn2MdImPMXcqn/wl1dppMn2MBPoY iexjpLE+RuL5GAn2MdIlPsbcqXzyl1Srp5mZPkYGfYxM9jHyWB8j83yMDPsY+RIfY+5UPvnbPbbT ZPoYGfQxMtnHyGN9jMzzMTLsY+RLfIy5U/nkb9nYTpPpY2TQx8hkHyOP9TEyz8fIsI+RL/ExZKfy yd92sZ5mYfoYBfQxCtnHKGN9jMLzMQrsY5RLfAzZqXwuiAxcmD5GAX2MQvYxylgfo/B8jAL7GOUS H0OeK5+JvyxaO02mj1FAH6OQfYwy1scoPB+jwD5GucTHkOfKZ2IvbbZM1nK2MNxzbDC2PAWMPTEK jP1wVDC2GALG7QiHZTiK9DJNUtzmwttirDU+SGmE14u+TY33KWQlkjM++zkHcZsbH6XyPkzRhkmY MC0UqM5V08Re2mx9iYwayJ7DoCLVQPbEUKjoNZAtBkN1v7SZ03EpaXKHBRbUNKfFL9m5tBSdblCV ei0Lm5O0sd7O7GF5jiy0d7H+WM6zzIaytos8V1wTe2mz9hI5WxjuOQgq2haGe2IkVIwtDLcYCtVp C8M5iVQ7EuWwYFBcnJLZh9lGJ273tm9J1/ysF2HTkoWxN6ZKDBU/qa2Iiy1ToDDVKdaydzBcXyGj 9LLnMKZIpZc9MZQpeulli8FM3a9s5qMIyS/zjSltjc1hUlHOU71bHu5+OmpdvFJLEMWUxd+gcjpN qixRFL20rUIoUHUKvewdDNeXyCjb7DkMKlLZZk8MhYpettliMFT3K5st3pglL/IGlSqxfqaxNjtS TSaFG1TKBT/FenMSU05eqxtUVmcf5bR47Yuq3TAKVJ0iMXsHw/YSOTsY7jkIKtoOhntiJFSMHQy3 GArVeQdDm3Ptjh9aKjX52i2Kc1sMpLj5AJXOWc0hZL2k2orJ5bBcXoy1w610TpNxOswUqDoFZvYO hutLZJSL9hwGFalctCeGQkUvF20xGKr7HQylUKnIsByhsk4bodzihY/LoU8lRJa1K1VEEVE5dehT zbJ11GcrliJCbawIUC3n4nQySD+ds4PhnsOgIpWa9sRQqOilpi0GQ3W/g2FQRhhvww2qYMNSGxtX cZN2jocvf2r2OZZQvNMuB3XoU4USbf1mKUK2ehaJBNW5sJ3YS6y1l8jZwXDPQVDRdjDcEyOhYuxg uMVQqO53MPzzV4tdOkVx9vJs60tkTDnacxhUpClHe2IoVPQpR1sMhso8HVSdgjp7abf1JTKmK+05 DCrSdKU9MRQq+nSlLQZDFZ8Oqk5Bnb3rYnuJnF0X9xwEFW3XxT0xEirGrotbDIXqftfFJ4CqU1Bn 79i4vkTGNKk9h0FFmia1J4ZCRZ8mtcVgqPTTQXWuqCf2bo/rS2RMsdpzGFSkKVZ7YihU9ClWWwyG KjwdVOeKemLvFNleImenyD0HQUXbKXJPjISKsVPkFkOhut8pcoy9spyr4skjnW3Obo97DgODND1r TwwFgz49a4vBYDyanvXN3FqVOJlYv+tNWVgtDgMqpP0blnO1O3mkE52ZqlMGVSfaLo57YigsPNUJ 28XxdoTRrYjqVKwd0hHm7MS45zAwyLrSwJ0Y98OxwIB1pfudGAeB0ak6O6Qzy9lNcc9BYNB2U9wT I8Fg7Ka4xVAw7ndTHARGp3JskQ4pZ0fEPYeBQfaGBu6IuB+OBQbsDd3viHgYL731O+bFmKCd9ovN ZuFuHKU6JWHLle/Xc2cKQchWh1ueTMtYIYix1eEWg2m5ZB6X6pR1DSDfu4kn9bQcAkbNE8FoiYFg tMMxwKgxEIz9CM9ULFGdsq4B5Hs38aSelsOgoko9LTEUKpbUU2MwVPdSz58P1bmsW//L7wm7iSf1 tBwGFVXqaYmhULGknhqDobqXev58qDpl3QL0op3gST0tB0ElqFJPS4yESrCknhpDoRJPJ/WoTkm4 AJ1tJ3hST8thUFGlnpYYChVL6qkxGKqnk3pUp3SckY664Ek9LYdBRZV6WmIoVCypp8ZgqJ5O6tGd snNGOuozT+ppOQiqmSr1tMRIqGaW1FNjKFTz00k9ulOyzoB972ae1NNyGFRUqaclhkLFknpqDIbq 6aQe3Sl3B8C+dzNP6mk5DCqq1NMSQ6FiST01BkP1dFKP7lTFA2DfO8mTeloOgkpSpZ6WGAmVZEk9 NYZCJaeng+pcUc9QRV3yhKCWw6CiCkEtMRQqlhBUYzBU6umgOlfUM1RRlzxxqOUwqKjiUEsMhYol DtUYDNX9Gkl/PlSdijoyTdZJnnTUchhUVOmoJYZCxZKOagyG6n6NpD8fqnNFPc9IR33hCUstB0G1 UIWllhgJ1cISlmoMhWq5RFjS56p4fYOBW9jCE5ZaDgODKiy1xFAwWMJSjcFgPBSWXj9UoYwqyR4W Yvv+T/4enb/9bTuB8xHeS9W5LJ4FoN+7hSc2tRxGFVVsaomhVLHEphqDqXokNr0+yQetzGOqkl7e oOr1CO+kypzr4lkAnr5TTCtKgVaUIltRaqwVpXhWlIKtKHXJ6tbmXNvOE+DpO8U0mxRoNimy2aTG mk2KZzYp2GxST2c2mXNtO0O1bcU0mxRoNimy2aTGmk2KZzYp2GxST2c2mXNtO0O1bc00mzRoNmmy 2aTHmk2aZzZp2GzSl+xybjr1aWTCqtNMO0mDdpIm20l6rJ2keXaShu0kfcku56ZTY0YmpzrNNIw0 aBhpsmGkxxpGmmcYadgw0pfscm46dWJkcqozTEvIgJaQIVtCZqwlZHiWkIEtIXPJLuemU+tFJqc6 wzR9DGj6GLLpY8aaPoZn+hjY9DFPZ/qYTp0YmdjqDNP0MaDpY8imjxlr+hie6WNg08dcskO66ZR6 kTmszjJtHQvaOpZs69ixto7l2ToWtnXs09k69lzpLRLp9FqmrWNBW8eSbR071taxPFvHwraORXdX /+mXn76uWIgXLKbp+5LcHKRbCrfLK6SXZfaCisWek1ws9rx5zUdfn/vxy09//8V/+vLdYb77SUPG nJDZjxZfj/af/qevLfLdgV4erMdwD46QHmEaYz8R6itoF+96cL2lbv+uycWeiNySsZU1/v1lfeKy n+fLP2vOpH4s15j/R9LLx/rs6D992qLfPdTecmn7+ZVo/+Xry3M/NtJ+/WWn+vyDeqzyO4dqaO+B D//N/I8P//Nzzv/b//N/fPg//6//2Ua+1lfz328t6GyTif7WbtpZzKZ4eVhbf3/IP7gg4qdf48/r FTG/XBHz3aS2ekV4xx6+n9tGFCp76hWx5xqDv5av62nubdP+m+8e/Y+2a8WDg1Abqy2GNFa3IwCN VfezcefPZh0Ef8+Zvu9X3M0U2n5Fu08O/BXi/CvWsbWBv+LUrNdf0b6wDvwV8vQr7DqjeOCvWDq/ onWEB/4KdfoVnt0tatwry7vat1FH8GpnjDVuMfRqvx9rHHK132mb62czD75OTOdXDL5OzvcTLwZf J+dm0Yux14k5N4t+arL/wF9xbhb91NTvgb/i3Cz61gke+SvOzaLXYz9uc24WvR78cXeaRTX44+5c 3Wrwx925utXgj7tzda+7wA/8FZ2re5mGvlG2c3XLsf0q27m6/diP23aubj/247adq9uN/bht5+p2 gz/uztVtB3/cnavbju1G287VbcZ2D2zn6jZjuwe2c3XnsR+361zdaezH7TpXdxr7cbvO1R3Hftyu c3XHsR+361zdYWz3wHWu7jC2e+A6V7cf2z1w56s7iMEf9/nqDmLwx32+usM09OOW0/nqDtPQj1tO nau7DP245bkOXX/F0O6BnDpXdxnaPZBT5+rOQ7sHcjpf3UEBH/fnf//29df1d8jX33EugYbJc2d2 z2ER2aVAL4qsucQdFNjyWbx/rGhP1DPd3pWP6acvv63jBcdiTP9n7Rzcg9fAqMm0GFaTeTkCUpPp snFq0Sob3Ana62lmJhsZZCOT2cgHNj7nrz0sbg+337w8+s0sIqAJSLcjjCbiXDGvRHBnV7fTLJJH RM1BRBRJJaImhhBR6JOHthhKREEnD/WJOPUXgpi4ZtN6mpZJhAWJsGQi7CAiOJX8FoOJeLOS/822 kultjLV8/+eBlNIH5dTrq6BwTaf17BMTlASCksigpEGg0GcCbTEYlEQEhcLFqasepgJ0QOdpZnHR cggXNU/koiVGcNGOw+CixkAu9iMMv6WcvllVIoBu5zwZJhEGJMKQiTCDiKDP4tliMBHoLJ4+Eacv wpUIoNs5T5FJRASJiGQi4iAi6NN3thhMBDp9p0/EqSoZpsydXtxOUwgeETUHESEElYiaGEKEoM/b 2WIoEeLNeTtAb+JUSq5c8I2+erKayYUGudBkLvQgLujTdrYYzIV+wMW3BWsrOiXOxJ0cvJ5oYDIR QCYCmYkwiAn6rJstBjPxaNbNN4ExcS5tigXpY84Tj4mag5iYJyoT8zSGiZk+4WaLoUzMj/a8Bpk4 e2+VCaSXOSsmEwpkQpGZUIOY4CjoLQYzASvoXSI6xU2J9DJnzyTCg0R4MhF+EBH0pWm3GEzEoz2t 0VaiU8eUSA9zLkwmCshEITNRBjFBX1l2i8FMoNtZ94k4VzDFjPQv5cIjouYgIuRCJaImhhAh6cvC bjGUCPloWdjtSXYSk7Hyu9U/b6MhusyTmbXSs05G6dxbp/FwhNfff597L2fnuqiY+fNX6ovnqRot h3FGVTVaYgxnLEGjxmDO3hQ07ofatDD1rqJj/f9FF7VQGqRzuVQI7kTf9ex53kbLYaBQvY2WGAMK y9uoMRiUhwvHmrbf1Osf7I51rp4KwV3jpp31wtM4Wg4CZKFqHC0xBJCFpXHUGArI8kjjGArIuYwq LNLJXXhWR8thgFCtjpYYAwjL6qgxGJBHVsdQQM41VWGQPu/CszlaDgOEanO0xBhAWDZHjcGAPLI5 hgLSKbAapLOqmFqHArUORdY61CCtQ/G0DgVrHeqR1jESkPM8xCA00klVTMtDgZaHIlseapDloXiW h4ItD/XI8hgKSKf4qpFOqmJKHwqUPhRZ+lCDpA/Fkz4ULH2oR9LHUEA6lVgFqMezZjogGnRANNkB 0YMcEM1zQDTsgOhHDshQQDqFWQUox7NmyiAalEE0WQbRg2QQzZNBNCyD6IcyyEhAOhXVBXGPNdMM 0aAZoslmiB5khmieGaJhM0Q/NENGAtKppAakk2qYmogBNRFD1kTMIE3E8DQRA2si5qEmMhKQTiU1 IJ1Uw3RGDOiMGLIzYgY5I4bnjBjYGTFvOiNvTH+KFFA6FVWPdFYNUyUxoEpiyCqJGaSSGJ5KYmCV xDxUSUa2JJ2Kqkc6q4bplRjQKzFkr8QM8koMzysxsFdi3vRK+Eb7edGgIBzSR7VMu8SCdokl2yV2 kF1ieXaJhe0Se2eXfDO2Le7/3jtNO8i8LG1ey/dqyfl37U/7IYV2WjF//JI/lY//WX/rb/nzlur+ qL354XS46LMUS+3m/lxDn3P+5evLQQ4PtOhpces9WkFpKzu3V71/Wvu/2pv9OHN4p+KdiVP/WDEJ Z8rhTfvuAMp33r/1J1pu79967j59DP/+mr+8vim3h+rJ+fN7+3KIFv97pe1f/tM/82v69ZEalqcV u/ewWcP1i8PP6yfwGn59pLGbTmuL7+ly+FiPV8Ddg+0Ezhf8egjTLrz69C9fffx5D72uAN75QT1U OS0jvh8qrGezN1mvJ7L/u53D9CBpt3fhS6yf69d65eX4mj481t4J/wARJ45HqCgewDw/Xo+0nK/r 7UDq5S397fhWNjrzg8/Q5e1quGV+voVmdW4x11So2NwSLxdAfPBJR0W/bFrmcNnIzmUzBReUmx9c Nml+dNnUn6yXTXuhd03AyxssH7zBNfrDv37zv/y0f8bb39tnqx+8U7m+jkPg5fnLo6c3DCv0n37+ /Hopv/xzPa8HVwL9RrDGoBvB6xGOzVvM0Sqz3D4u21ZjMHKZ3ZKiFTdJ+dvsU57yEn37xqWndHDE kpxKiiJqrYJ0ev24fvzxxw/58+dfP39orULrX5TPv/7jQ+321/fmy4e7W4b9UJ8/8rahZ/Zt412N 7PLo9/7VyP7VyP7VyP7VyP4hjez7vrudV2Ot392QWYaWaWxb0Ni2ZGPbDjK2Lc/YtrCxbR8Z2yOL Pue1dEPb4wQAhGlqW9DUtmRT2w4ytS3P1LawqW3/CFP7vBJyqPd4oPrjmKa2A01tRza13SBT2/FM bQeb2u6PMLXP61gHUZCysWOa2g40tR3Z1HaDTG3HM7UdbGq7P8LUPq9CHkRG6seOaWo70NR2ZFPb DTK1Hc/UdrCp7f4IU/u8hnwFBOmkeqap7UFT25NNbT/I1PY8U9vDprb/I0zt8w4AFRCkk+qZprYH TW1PNrX9IFPb80xtD5va/o8wtc/7NwSRkE6qZ5raHjS1PdnU9oNMbc8ztT1savs/wtQ+775RAUGm EwamqR1AUzuQTe0wyNQOPFM7wKZ2+CNM7fPeKUFEZDphYJraATS1A9nUDoNM7cAztQNsaoc/wNRe zjvfVECQ6YSBaWoH0NQOZFM7DDK1A8/UDrCpHf4AU3s571sUZol0UiPT1I6gqR3JpnYcZGpHnqkd YVM7/gGm9nLedSrMM9JJjUxTO4KmdiSb2nGQqR15pnaETe34yNQeCsi5kjrPSCc1Mg3tCBrakWxo x0GGduQZ2hE2tOMjQxsXcJfzRm9hFkjfNDLF7AiK2ZEsZsdBYnbkidkRFrPjdWL2ct6dL7Q11flc JKaYnUAxO5HF7DRIzE48MTvBYnZ6tOzfCC7O5dJ5QuYMJqb0kUDpI5GljzRI+kg86SPB0keC91H0 /1hxWG44dGpfBZn4lWXyHBxqLiI4ZFn7WiQcaqKe6bffvn5Mv/6SP37dQTg8ULPm7HNuUQ4BGR60 z3eD9t90zLOW4t1TNqapJdremb87ZeP1ae9i6FwemzXSBcmWyZAFGbJkhiyfId64fobH9fPduP63 g4G4Pm1dFFY52V9Wdnm4juwWef2Vy3vYEefKWVv2A2AnMdlJIDuJzE7is8Mb8s/wkH++G/L/Vlvd kETqrEy8/kSZ+/lQLz+xs71rnOpPVKwtj8/TA8ReDvn9mR1y7yKuU4pTSAeozDziag4irsxU4mqC S1zhOQQFdgjKm5v4va8vfOagU3FTyNoIxTA5MCAHhsyB4XPAUwUKrAqUO1Xgm5fe7R/8/c1LO+H7 N6/bzSqVtmbso3vZfoRTQ/OSexdgnYrdgnStS2QCFkHAIhmwyAeMpxoUWDUob+4EyG1oOhW6BXBg 5SRYHLQcwkHNEzloCSYHLcrgoMZADvYjHGfFt2ZGvndW/LajQhF6+t1Z8Yen9fdkiHFy3Ux28/S+ 9qdTAZSAWisnzeROg9xpMneazx1LVKgxmLuHosLtw3eTeeu+5o1R9+idjtDhasu9i6tOBVECRq6s nXoeVwHkKpC5CnyuWH5DjcFcPfIb4Pva2auUBehASzHxOKg5iAMxUTmoiZ0D/+Xfv8TvQHh5pJHw 6NdxSBCoyLAf4diFLmVaZt9palLSpljf+fK+/sTFTtujazMvpvio7Xk55Kntecm9gzl1WoZq0Yqv SURfO32NHSJzWy6zmVvzeSYwtyXixtyX6H/5JX/+bvji/Hj73ao3Y7xwxjvXGDgFutyPd3671a0P 296tf93ZEv3G6L/Kl5/z519WONQrHPN9g7QooYGOdv0SkxyjQWq5ADRILV9e819/+kdO/+lfPqf7 R/+ju+bEepBc3+yP8V+35378EvajnB5uJ2PPtGzHqW3df1WoPv72+deY0z8/75/694+1I7gHL4fT 4NUY2ODtRwDGyx5wdmqElFBAx7p+42VyVkDOygjOyiDOCswZy+OoMZgzdOPGB5zdj6s1zoCOdiwL 42a35fg3uzVPutltiXqmX/2Xn/+/f+Z/vixv8/HTr7++rKPS+1E9Xjyv3bIdjkNGQU2O/QjDyZD3 o2aNDEAnjsUlzSLDJQuR4Q5dd24LVNzaf4dboHqcdX2if5aP+V8/xe/9kNPD9TjSnlfx2Y7Dog31 Q/YjjKftPGJWFOAmpzmz2qG0DrLyaat5YjvUEgPboXY4Bhk1BpKxH2E8GecxtKIAKTlJySOj5iAy pKSSURMjyZAso6jGUDLko2VAQDLOg19lAZygJC2TDAuSYclk2LFksDyhGoPJeLT+B0jG/XDYomUB erVJJiYZCSQjkclIY8lgWUA1BpPxaOEPkIz7AatGBtCrTcvMI6PmIDKWmUpGTYwkY2HZOjWGkrG8 aevwyLgfcqrfdxzSA10MkwwDkmHIZJixZLD8nRqDyXi01AdIRqfm65Ae6BKZZESQjEgmI44lgyXe 1BhMxpviDY+MTpXWIj1QJXhk1BxEhhJUMmpiJBmKpeLUGEqGerS4B0hGp65qAec8Kc0kQ4NkaDIZ eiwZLFmmxmAyHskyGBlLp65qADc8qcAkI4BkBDIZYSwZLN2lxmAy3tRdeGTc10AbGYDskvTEI6Pm IDL0RCWjJkaSoVn6S42hZOhH63iAZNzXQBe9SKQHqhWTDAWSochkqLFksBbwqDGYjDe3WuSRcV8D rWTMSA9UeyYZHiTDk8nwY8lgGSQ1BpNxjUGynGugy4z0QDVHY9tyGBk0jW1LDCWD5XzUGEzGNc7H cq6BLgLpgRqe89FyEBmG6ny0xEgyDMv5qDGUDHON87GcaqCVDKQHahyTDAeS4chkuLFksPyMGoPJ uMbPWE41UL1MwETDZJh+hgH9DEP2M8xYP8Pw/AwD+xnmGj9jOevyywQY0cky/QwL+hmW7GfYsX6G 5fkZFvYz7DV+xnKqgWpZAIc5WaafYUE/w5L9DDvWz7A8P8PCfoa9xs9QpxqoXjTSA7VMP8OCfoYl +xl2rJ9heX6Ghf0Me42foU410EoG0gN1TD/DgX6GI/sZbqyf4Xh+hoP9DHeNn6E6NVCF9EAd089w oJ/hyH6GG+tnOJ6f4WA/w13jZ6hODVQhPVDH9DMc6Gc4sp/hxvoZjudnONjPcNf4GapTA12QHqhn +hke9DM82c/wY/0Mz/MzPOxn+Gv8DNWpgS6IIeyZfoYH/QxP9jP8WD/D8/wMD/sZ/ho/Q3VqoAti CHumn+FBP8OT/Qw/1s/wPD/Dw36Gv8bPUJ0aqEQM4cD0MwLoZwSynxHG+hmB52cE2M8Ib/oZ27Nt fXZeFrdEy0OlUxT1SJc0MIWNAAobgSxshLHCRuAJGwEWNsI1woY6F0UXh3RJA1PYCKCwEcjCRhgr bASesBFgYSNcI2zoTlHUIV3SwBQ2AihsBLKwEcYKG4EnbARY2AjXCBu6UxR1SJc0MoWNCAobkSxs xLHCRuQJGxEWNuL7hY29BzILNfllcu731/i8Pe3dNHUKqRbpxkam5BFBySOSJY84VvKIPMkjwpJH vEby0J1CqkU048iUPCIoeUSy5BHHSh6RJ3lEWPKI10geulNINYhmnJiSRwIlj0SWPNJYySPxJI8E Sx7pGslDdwqpBtGME1PySKDkkciSRxoreSSe5JFgySNdI3noTiE1I73WxJQ8Eih5JLLkkcZKHokn eSRY8kjXSB66U0hNSA80MyWPDEoemSx55LGSR+ZJHhmWPPI1kofu1E0T0gPNTMkjg5JHJkseeazk kXmSR4Ylj3yN5KE7ddOI9EAzU/LIoOSRyZJHHit5ZJ7kkWHJI18jeZhO3TQiPdDClDwKKHkUsuRR xkoehSd5FFjyKNdIHqZTNw2IZlyYkkcBJY9CljzKWMmj8CSPAkse5RrJw3RqoAHRjAtT8iig5FHI kkcZK3kUnuRRYMmjPJI8XgvjteeY6rfQR6jcbx9yC7ybok691LOV5OjzxBmX2XLITiE1T9wppCWG UbQdjrFdSI2B24XsRzhsF1KCnlWZ3Q0nN+nspA5u8YtQxdxUkfqmWT27aXF+aqP4tyEcoWQpaXLF TsrnaEht07nW2tabA6jijM9sOYwq2vjMlhhKFWN8Zo3BVN2Nz3wTWky+3JiSWU3LNLvQdsUS6oCU m4pYNzyfZP2JOSBVX4uYhJx9FnKKJKLONVol2D3k9vo44zpbDiOKNq6zJYYSxRjXWWMwUfmeqBBC Ue6wu5HxKge5ROOi92sDtjMVrKpfkpKx1jkby2GkOWjvprAsPs2Ti1qSoDqXd9XE7lzHRjUPKoEM Ca15KlRi4JDQdjgOVAIbEno9wgGqlJtMlvINqpdHDj/58vAnof8TElXn0rCa2B3z9ho5w0lbDqOK Npy0JYZSxRhOWmMwVfa+S5Xesl2mwx8SLZ1ycUE64IIzxLTlMFpoQ0xbYigtjCGmNQbTcjfE9G2Z VEpq0ocO+DznkJY0z1Its063Nkh4FUvSpQ1GtGfcqLJTUEn4qEVUUZaZRFWn1FzY8nZ9jfMcJYeq mlMIVTXvsJ2PtoMkeOej9ThpmX74NPvP8WPJ9V077nx0eriBb1T/OCxSZ2zI6/UIx027a7eqeCdv pHrvrXezsSKYOOf50K9XzgQdXQnBeK/kof1zIURZv/gbJaSQjkKq7ZS+M1smb6+RM1y25aD2b6YN l22Jke3fzBguW2MwVdcMl9lO6TuzZfJ2nrE1AwwyYiwQGTHJ13z09bkfv/z091/8py/fHea7n6yN R5+aeNhLbmuzfvr7dwd6ebAe47zx2nYEtzWH38ePj7RXo8qDdH036kf28b/yv/IvXz/eGuS7B1t7 HPuHyKK+hH9/2Z9bMcuf95fw/YP1EPLBa+DBjo0Avh7h0IQGL7Qu+lC8VYtyk9Fe+xj1JJZbE+rt 5O08T/VLZpy0urWgUWsZdZiUs2HyUyJcJ+buOpG1L1pm7rfYZbLOMqbj7Dl2C7rlKS3onhjVgu6H o0K1xRCobkd4pm+xxs4dqrjfYtfXyJjKs+cwqkhTefbEUKroU3m2GEzV/T7iT0CVPFMluN9222t0 jOGmPQdR5UjDTXtiJFWOPty0xVCq3CXrthq7dMjgfmNdz5MxZLTnMDJIQ0Z7YigZ9CGjLQaTcT9k 9ATtjTpTNXG/Xa6vkTFstOcwqkjDRntiKFX0YaMtBlN1P2z0BFTpDlXcb6btNXrGuNGeg6jypHGj PTGSKk8fN9piKFX+kqlExpoOGdwJI+t5MsZ+9hxGBmnsZ08MJYM+9rPFYDLux36eoL2xZ6oc0mv2 jDGiPYdRRRoj2hNDqaKPEW0xmKr7MaInoMp1qEJ63IExhWnPQVQF0hSmPTGSqkAfz9liKFXhfjzn z6fKTWeqLNLjDozxnD2HUUUaz9kTQ6mij+dsMZgq83xUdWrcFulxB8bUqT2HUUWaOrUnhlJFHzjZ YjBV9wMnRmqlpT7of/P0/f9uVG3PNJOR86SLWW5UrUfJ2ulYf5J1IVHVqXFbpLceGdOu9hxEVSRN u9oTI6mK9GlXWwylKorna6s6Ne6FOzV8fY2MKVt7DqOKNGVrTwylij5la4vBVF0yZcu4To174U4N X8+TMWVrz2FkkKZs7YmhZNCnbG0xmIxL1uU1rlOnltwJM+08E2Nd3j0HkZFI6/LuiZFkJPq6vFsM JSNNz3cnOtWpxSSQXnNiLOG75zCqSEv47omhVNGX8N1iMFXq+ag61bgrVUivOTF9owT6RonsG6Wx vlHi+UYJ9o3S8/lGrlPjjkivOTF9owT6RonsG6WxvlHi+UYJ9o3S8/lGrlPjjkiPOzN9owz6Rpns G+WxvlHm+UYZ9o3yI9/o9Ul2iV6WZG8wleOflPzvrppwOMI7sbJTp8gdkO56ZspKGZSVMllWymNl pcyTlTIsK+Wnk5Xs1ClyB+4c9/U1MmWlDMpKmSwr5bGyUubJShmWlfLTyUp26hS5PTI9oDBlpQLK SoUsK5WxslLhyUoFlpXK001yt1OnyO2R6QGFKToVUHQqZNGpjBWdCk90KrDoVJ5OdLLTqUAuJoV0 1wtTdCqg6FTIolMZKzoVnuhUYNGpPJ3oZKdTcb1+akBv3U880anlEKpqnkhVSwykqh2OQVWNgVTt R3guqjrF9QXorfuJJzq1HEYVVXRqiaFUsUSnGoOpejrRyU6d4roEeut+4olOLYdRRRWdWmIoVSzR qcZgqi5ZI9pOpwJ5JQPocXvBk5VaDiJDUGWllhhJhmDJSjWGkiEerRH9TZTDikB5dlN977wtORs1 3xqWmQbMqfYtphmYNeAFz0NqOQwYqofUEkOBYXlINQYD88hD+h4Y75dQ+92TNW11CsUFRpyq2hUY YEKAFzw9qeUwYKh6UksMBYalJ9UYDMwlepIVp8q0mAQg9fuZpye1HETGTNWTWmIkGTNLT6oxlIz5 zW3DeWScqstiQqrLfuYpRi2HkUFVjFpiKBksxajGYDIu2SXcilOFuJKB9FdnnibUchgZVE2oJYaS wdKEagwm45Jdwq3oVHmR6ax+5qk+LYeRQVV9WmIoGSzVp8ZgMp5O9bGiU+VFprN6yVN9Wg6iSlJV n5YYSZVkqT41hlIl73eysNmXLA/r6GalTJy0mUR0LtvD6qTeOV1CdDpkPUd/+NZcjx1jnJzPJusy LySoOkVeZDarlzzRp+UwqKiiT0sMhYol+tQYDNW96CPjvCi9HNa8DVIWb2W9aweppyncqIpTnKPM USc7m1Dsgapsan9GaSOkyNJS5h1a0SnyIrNZveSJPi2HUUUVfVpiKFUs0aetOoNSdRJ9rF6clodN d+w8yxK1UFHVH3x3A5Qqhrasx2zMUsKBKmUXV5RxRdgipKRsumNFp0CMzGb1C0/0aTmIqoUq+rTE SKoWluhTYyhVy73os6yTP9Nx252Q7aS00Ev9X9AH6TU7613bNK82YSnZeFhdtiyh3gOnKJdJ2OhJ VHWqyArw8v3CE31aDqOKKvq0xFCqWKJPjcFU3Ys+ailqCepwB9RJ1/6TkmnS9e5UDlRJE62VeVK5 IrfEw64XKkwheBnzVJOqkPpVc6fUnJHO+sITfVoOo4oq+rTEUKpYok+NwVSddr0QcZndujbDTtVS Gxu92NnNYgpuijeq6rUwVajmyUrbths4DFlkL0Rtrial/GwTqa2aO2XqhPTWFVP0UaDoo8iijxor +iie6KNg0Uc9n+gzd0rcCemtK6boo0DRR5FFHzVW9FE80UfBoo96PtFn7pTHE9JbV0zRR4GijyKL Pmqs6KN4oo+CRR91v6LRE1DVKa0js2i9ZkpCGpSENFkS0mMlIc2ThDQsCemnW9HIzp3SOjKL1mum SaRBk0iTTSI91iTSPJNIwyaR1s9HVae2jkyi9ZqpG2lQN9Jk3UiP1Y00TzfSsG6kw/NR1amtI5No vWGqSgZUlQxZVTJjVSXDU5UMrCqZp1tJyc7n2rrQSG/dMDUnA2pOhqw5mbGak+FpTgbWnMw1mtN8 ro8LgfS4DVNzMqDmZMiakxmrORme5mRgzcncr4YkfoeNb3PUXlsKIvJc7BYC6T4bpu9kQN/JkH0n M9Z3MjzfycC+kykPGo+Kw1yWw0rF678PI7rOhH1XcNJgrjyXssWEdI4t02ayoM1kyTaTHWszWZ7N ZGGbyd7bTHO2XiobDgMkdnu6EdnmYt1BPBFTKVO2xtvUnnUQT+y82MW6+h/SjUqey9hN4geIYqpM FlSZLFllsmNVJstTmSysMln3qBWKUygNlKQaGAcnQFgZtUi115WsOrZLUbZnuskkGxpgJJQ6teuC SP+W6S9Z0F+yZH/JjvWXLM9fsrC/ZO/9pWLtwQfYmChlnkRttSZ5IGptqBa78RMPu8DPi47tMWFz tHKhdZE6deuCTBZwTHfJge6SI7tLbqy75HjukoPdJXfvLtlFlOlgLplop/bsPKfK1HECY7bL3Ijb O0qHznY2Tmfr5hxaMs0kpjpV64xMM3BMc8mB5pIjm0turLnkeOaSg80lZy/5zi7PlWfhkM61Y9pH DrSPHNk+cmPtI8ezjxxsH7nnW2ZInivPwiEdbM+0jzxoH3myfeTH2keeZx952D7yz2cfyU7l2SJ9 bc+0jzxoH3myfeTH2keeZx952D7yz2cfyU7V2iL9bc+0jzxoH3myfeTH2keeZx952D7yz2cfLZ1C t0F63IFpHwXQPgpk+yiMtY8Czz4KsH0Uns8+WjqlcIPMFQhM+yiA9lEg20dhrH0UePZRgO2j8Hz2 0dIph2tkrkBg2kcBtI8C2T4KY+2jwLOPAmwfhWsWO1rO1W2hEd8/Mg2iCBpEkWwQxbEGUeQZRBE2 iOK9QZR1UGY5LCSgRRtMlrpklZwPhwJ3MYufZyFinIKYvbg1N1bp4mZttJ70FDJpwttyLnDPAulw R6ZAFEGBKJIFojhWIIo8gSjCAlG834qtAmJLMYe5uUZak2rfJGYnF6cO8928Fza7aEXWdj4MmpRJ eRttWJxV86JoDdW5wD1PSHc7MtWjCKpHkawexbHqUeSpRxFWj+K9euRCqpQodVzpcXFziMZ54Zxe DpMojRChJKuM19KJY0slbJznoMUc4yyLI0F1ro3PE9LbjkxZKYKyUiTLSnGsrBR5slKEZaV4vzhT bXSkW/RhZq7LtuKhdPDJz8UdettL0VLUpkVGFZwR4XD7czYnGYPT9f2TjjS+u3Rq4wXpbSemzpRA nSmRdaY0VmdKPJ0pwTpTuteZ3NKckxIPLpxXVnuvZi+zVOqwOlPUSk6LLtMUcwjHFU/8rJOy0alg wpQUab730qmNF6SnnphKUwKVpkRWmtJYpSnxlKYEK03pfnWmqOw8x+MqAn42cppKzJNtswoPN8Ag Xf3CEsucjZdJHyQ5qXSepqBVCKLeGjWJqk5tvCCuf2LaTQm0mxLZbkpj7abEs5sSbDelfEllQHXq 2x5x/TPTUsqgpZTJllIeayllnqWUYUspP99WaqpT3/bI9IDM9JQy6CllsqeUx3pKmecpZdhTys+3 lZo617fnBelxZ6bjlEHHKZMdpzzWcco8xynDjlN+PsdJnWvj84L0uAvTcSqg41TIjlMZ6zgVnuNU YMepPJ/jpDrF8QXpcRem41RAx6mQHacy1nEqPMepwI5TeT7HSXXK4xLprRem41RAx6mQHacy1nEq PMepwI5TeT7HSXXq4xLorYeJ5zi1HEJVzROpaomBVLXDMaiqMZCq/QjPRdW5Pj7PwIyEMPEcp5bD qKI6Ti0xlCqW41RjMFWP9moDa07nGvc8A7MKwsTzlFoOI4PqKbXEUDJYnlKNwWRc4ympc516FsDM gCB4nlLLQWQIqqfUEiPJECxPKQjUU9qPMJwMfa5Tzxbo+QbBk41aDiODKhu1xFAyWLJRjcFkXLNa kT7XmmeL9F4FTxlqOYwMqjLUEkPJYClDNQaTcc2mbLpTLzZID1TwvJ+Ww8igej8tMZQMlvdTYzAZ jxYpAsno1HwN0gOdee5Oy0FkzFR3pyVGkjGz3J0aQ8mYl2vI6NRtNdIDnXn+TcthZFD9m5YYSgbL v6kxmIx7/+a29t361y+3v764pSRCOjVYDWjvYea5NC2HEUJ1aVpiKCEsl6bGYELuVwpSqi0Ne+Ak hJjrp5rbMnmznszBUA5RhFB8lrEtJGQOC7uoOE1lktJIJ00gyaS6U4JVgPceJE/DaTkIKknVcFpi JFSSpeG07RJBqOS9hiO010vU6bAElbciudpALcrVDtphuaCyiBzrT2qrn4wv6jCbIqkwixyylVmY RNL+dKcEqwDxPUiehtNyGFVUDaclhlLF0nBqDKbqtNGZTUbbA1R5mZWfcgWqxGyO63YuFb9opkXr HGa/ThV7cUmdW4yZTP1OPvk50u5/5+qtRBZdDJJn4bQcBhXVwmmJoVCxLJwag6G6t3B0/UY0mXRc fdFmEaJybftGqe1h/9imh+ZFT0o4HY08TPzKU7HauRBEJc6QFjbTncIvsvxiWHgSTstBUC1UCacl RkK1sCScGkOhWu4lHBtjvQWmQ1OlXFTBZqeUFGqJh8kU0puwLEuq774qszjc/5yd4qxj/d8UtY6k L3OmUzRGlmAMC0/CaTmMKqqE0xJDqWJJODUGU3Uv4Tih9DyHw2SKrLaNNrMyRSt9g0qryUZv0xy0 mb0/dNXrja+I+iXAlopk7dGToOrUm5E1GMPCc3BaDoOK6uC0xFCoWA5OjcFQPZ+DYzq1amT32KCY Do4CHRxFdnDUWAdH8RwcBTs46vkcHNOpcyO7xwbFdHAU6OAosoOjxjo4iufgKNjBUc+3zpDp1MiR 3WODYvo7CvR3FNnfUWP9HcXzdxTs76jn2+XMdOrqyO6xQTPdHw26P5rs/uix7o/muT8adn/08+1y Zs6FdYms3xg00xvSoDekyd6QHusNaZ43pGFvSN8vUvQEVJ0L6xJZvzFopnOkQedIk50jPdY50jzn SMPOkb5fpujPp8qey1USWb8xaKavpEFfSZN9JT3WV9I8X0nDvpK+X6foCag616sksvZjMEzXyYCu kyG7Tmas62R4rpOBXSdzv07RE1B1rldJZPfYYJielAE9KUP2pMxYT8rwPCkDe1Lm3pN6AqrO9SqJ 7DwbDNOtMqBbZchulRnrVhmeW2Vgt8rcu1VPQNW5XiWRzWqDZcpVFpSrLFmusmPlKsuTqywsV9nn W+PInutVEtnONlimXGVBucqS5So7Vq6yPLnKwnKVfb41juy5XlUC0lu3TLvKgnaVJdtVdqxdZXl2 lYXtKvt8axzZc72qBKS37ph6lQP1KkfWq9xYvcrx9CoH61XuXq9Sutgp6ePkBqVVDqrpfWWO6eDs BW+neuvWrkzFTOpggjorvczKSKl8VIq0obvtmKDI2pHBMfUqB+pVjqxXubF6lePpVQ7Wq9y9XiVt VGVeDhu65xh09NL6EOs3fH2gyqklS1eUm2btw3SQ9nxK1kgRcgpO25m0E7c9q6ASWTsyOKZf5UC/ ypH9KjfWr3I8v8rBfpW796tiZcFP9kDVEvK82DiVXLyOyh+o8vXmppROQSxSycMdUKdZF5ONL8XH nAOFKteprSP74wbP9Ks86Fd5sl/lx/pVnudXediv8vd+lazomGk+9Ktk23lcL3IubpbJHVzQYkqU QRVll1Dvk+nQVlVY5WJK/W4bpqAliapObR3ZHzd4pl/lQb/Kk/0qP9av8jy/ysN+lb/3q2qjUz9I f9gDwC2LkLLezKwWJsjDau0xTKltV2K1ScZ8twmOVm0fnMpbmwpYSJMBXae2juyPGzzTr/KgX+XJ fpUf61d5nl/lYb/K3/tVRWi7LEdtvX6HW6KPena10y7ioV+li5+TC8lK7ZRSh4nKQcxplsLmJQk3 B9J2Xa5TW0f2xw2B6VcF0K8KZL8qjPWrAs+vCrBfFe79KuXkEqWbjxOXtfezXNTiizTi0FbVpsst JRlZsvI6HHbBEVZKP/spiHoLdIE0G9Cda+ulIL31wPSrAuhXBbJfFcb6VYHnVwXYrwr3fpVd7GIW fdiwS5dlrv2vxcQlzUocthasfanaiZ8rUPNUfDrsVyKW4lKea389tXXySHNs3Lm2XjLSWw9MvyqA flUg+1VhrF8VeH5VgP2qcO9X2VlpI+xhjqmdpa03NC+EFnpa5zTvVNXv+3lxYRZeapFdPFCltF9E nCYVZx0mWlvVqa1npLcemH5VAP2qQParwli/KvD8qgD7VeHer8oqJTnZcphkGrUKeVkWaZPywh7m wxutnBaiWB1k/euhsuBtmorK0tTb4xRJ9SrXqa0npLcemX5VBP2qSPar4li/KvL8qgj7VfHer5pD 0kEfpy57n51I2QWvKibq0FtXJtsiljbXOSeR50NvvR0kujmWbFTQpKU73Lm2XhLSW49MvyqCflUk +1VxrF8VeX5VhP2qeO9X2ZTFPM2HykJbiWjSTrbFzOZ6wzu0VVKbqd77wlyknLw/jNjoWRnpdNs/ rthE2l3QnWvrJSKzISLTr4qgXxXJflUc61dFnl8VYb8q3vtVIdavPvWnN6pq333JQRk/xWyjO6x5 Jr2Us/IhWm2WqRxmxIuidUiyySRmdjOlX1X75R2qkNkQielXJdCvSmS/Ko31qxLPr0qwX5Wezq9y 07m2XgIyGyIx/aoE+lWJ7FelsX5V4vlVCfar0tP5VW76vrZezzIqqLaemH5VAv2qRPar0li/KvH8 qgT7Venp/Co3yQ5VSG89M/2qDPpVmexX5bF+Veb5VRn2q/LT7SHnpuVMFTR3OTP9qgz6VZnsV+Wx flXm+VUZ9qvy0+0h5ybVoQrprWemX5VBvyqT/ao81q/KPL8qw35Vfrr1q9ykz1RBc5cL068qoF9V yH5VGetXFZ5fVWC/qjzd+lVuMh2qkNkQhelXFdCvKmS/qoz1qwrPryqwX1Xu/SodlnmJi7lRVSY7 +aCXlOv9sn7vP4wuByPzpJXNxqdFHrz1ZO1cWnF9nuNiC6UK6ibboQqZDVGYflUB/apC9qvKWL+q 8PyqAvtV5d6vUtLUHxp3cBbSJJNNLnirk54Oi82aaEo21jhTIZqOhnFZ1DQtKphp9sFJyjigmzq1 9QLMhqj/YVHVcghV9T9EqlpiIFXtcAyq6n9AqvYjPNUdUEyntkpPQG89T7xxwJbDqKKOA7bEUKpY 44A1BlP1dOssOCHOd8AC9NbzxBsHbDmMKuo4YEsMpYo1DlhjMFVPt86CE53aegF661nwxgFbDqJK UMcBW2IkVYI1DlhjKFXi+cYBRae2XoDeeha8ccCWw6iijgO2xFCqWOOANQZT9XzjgKJTW0fmLmfB GwdsOYwq6jhgSwylijUOWGMwVc83Dig6tXVk7nKeeeOALQdRNVPHAVtiJFUzaxywxlCq5rtxwEOZ an+SyGqeSjqIxeX7PzeYVnb+9rftBM5HeDdWneI6Mnk5z7yBwJbDsKIOBLbEUKxYA4E1BmP1fAOB olNcRyYv55k3ENhyGFXUgcCWGEoVayCwxmCqnm8gUJyL63pBuuuSNxDYchBVkjoQ2BIjqZKsgcAa Q6mS4q1bYLKL8793Cyxv3QJfjvBurNwZK4n01yVvJLDlMKyoI4EtMRQr1khgjcFYPd1ONm7uVNcl 0l+XvJHAlsOooo4EtsRQqlgjgTUGUxUeNFav7HzfFL2XjHOFXM9Il3vhrZbQchAZC3W1hJYYScbC Wi2hxlAylukaMs5Vbj0j3eaFt+JBy2FkUFc8aImhZLBWPKgxmAx1DRnnSrUWwNyCvPBWLWg5jAzq qgUtMZQM1qoFeUFXLdiPMJ6Mc7VZC2B+QF54Kw+0HEYGdeWBlhhKBmvlgRqDySjXkHGuGOsJcPyz YlpDCrSGFNkaUmOtIcWzhhRsDanlGjLORV9tkB6oYpo/CjR/FNn8UWPNH8UzfxRs/ih3DRnnwq02 SA9UMe0dBdo7imzvqLH2juLZOwq2d1S+hoxO8VUjPVDNNHA0aOBosoGjxxo4mmfgaNjA0fIaMjr1 U430QDXTotGgRaPJFo0ea9FonkWjYYtG20vIkJ0aqEJ6oJppwmjQhNFkE0aPNWE0z4TRsAmj0xsD NrIIr3/XWYj/4/cHbF6P8G6sOgVUhYjrhqnCGFCFMWQVxoxVYQxPhTGwCmMeqTBgg9MpoC6IfG6Y NosBbRZDtlnMWJvF8GwWA9ssxlxDRqeAuiACuWEaKQY0UgzZSDFjjRTDM1IMbKSYeA0ZnQJqQbqv lmmVWNAqsWSrxI61SizPKrGwVWIfWSUgGZ0Caka6r5YphlhQDLFkMcSOFUMsTwyxsBhi9TVkdAqo GemBWqbcYUG5w5LlDjtW7rA8ucPCcoe9Ru6QnQJqQnqgjil3OFDucGS5w42VOxxP7nCw3OGukTtk p4CakB6oY8odDpQ7HFnucGPlDseTOxwsd7hr5A7ZKaAmxGt2TLnDgXKHI8sdbqzc4Xhyh4PlDneN 3LF0CqgWUZMdU+5woNzhyHKHGyt3OJ7c4WC5w91vK/Hnq8lLp35qETXZM8UQD4ohniyG+LFiiOeJ IR4WQ/z9cjLi3OIc6+t+8fHWBL0eSZgofL3TfF+YP/3Cl6f9kEI7t5g/fsmfysf/rL/6t/x5S3V/ 1D6IcH+4OfosxVI/h59r6HPOv3x9OcjhgRaND6Jp+iH6T5/aS99yL/9q7/jjzHHX2LtxjPrHikk4 U87v33YA5c/v3/YTLbf3bz33Skz499f85fVNuT30H23zof7Z1UO0+N/z14//8p/+mV/Tr4/UsDwh uIfNGg4+/rx+Aq/h10faFZTcg3Q5fKwb83cf6PZgO4HTxb8dwoh6Edanf/laf+Eeqh/IfkWef1AP VdKDQ4X1bPb26/VE9n+3c5geJO32LnyJ9XP9Wi+/HF/Th8faO+EfIOLE8QgVxQOY58frkZbTxb0f SL28pb8d38pGZ37wGbq8XQ23zM+30KxOreeWChWbW+LlAogPPumo6JdNyxy3xe1cNlNwQbn5wWWT 5keXTf3Jetm0F3rXBLy8wfLBG1yjP/zrN//LT/tnvP29fbb6wTuV6+s4BF6evzx6esOwQv/p58+v l/LLP9fzenAlkO8GWwy5G9yOcGzeYo5WmcOWoLbYIo1cZlfv1VYcFkKcW6c/L9EblYOeDtsX6ySn 2l8UUWsVpNs2Gfrxxx8/5M+ff/38obUKrbNRPv/6jw8+pfrefPlwd8uwH+rzR9429My+bbyrkV0e /d6/Gtm/Gtm/Gtm/Gtk/pJF97xe5s+5gkJWxs2d63B70uD3Z4/ZjPW7P87g97HH7+xUcb1/kvh1X K74BQ9o/azlrDwZZ5Tp7ps/tQZ/bk31uP9bn9jyf28M+t79fjXEwIWf9wSArVufA9LoD6HUHstcd xnrdged1B9jrDvcrK5b0O/Xnb22mz+0PiZWzEGEEUo4OTNM7gKZ3IJveYazpHXimd4BN73CN6b2c hYj6hQW4zwSm6R1A0zuQTe8w1vQOPNM7wKZ3eGh6t68Chz+He07tu37/g5fvIIkEz9mZMBMyHTEy fe4I+tyR7HPHsT535PncEfa54/0WZ79/C5qBe9BZozATMkMxMhXvCCrekax4x7GKd+Qp3hFWvOM1 ivfS0SgKMkMxMhXvCCrekax4x7GKd+Qp3hFWvOPzLTqozgqGgVZzTkw9PIF6eCLr4WmsHp54eniC 9fB0X7kb+w1anXUKA63MnJiaeAI18UTWxNNYTTzxNPEEa+LpGk1cdeqz0OLKiamJJ1ATT2RNPI3V xBNPE0+wJp6u0cRVpy4LLZCcmZp4BjXxTNbE81hNPPM08Qxr4vkaTVx16rHILvI5MzXxDGrimayJ 57GaeOZp4hnWxPM1mrjqVF+RneBzZmriGdTEM1kTz2M18czTxDOsiedrNHHVqb4iu7nnzNTEM6iJ Z7Imnsdq4pmniWdYE8/XrAGoOqVVZEf2XJiqdwFV70JWvctY1bvwVO8Cq97l0RqAl9fl1bnUaqFS a2HqJQXUSwpZLylj9ZLC00sKrJeUa5YJVOdSq4FKrYWplRRQKylkraSM1UoKTyspsFZS7rWSq6QB 3SmgIutrlIknmLQcwkrNE1lpiYGstMMxWKkxkJX9CMNbEd0pnCLra5SJp5O0HEYGVSdpiaFksHSS GoPJuEYn0Z3CKbK+Rpl4OknLYWRQdZKWGEoGSyepMZiM+y00xw666E4BFVlnowieM9JyECGC6oy0 xEhCBMsZqTGUEHHNGoC6U0BF1tkogieItBxGBlUQaYmhZLAEkRqDyXgkiHwzs1G6aKuKmbTQh9F/ OclJ2+NXXR1I2HSqq8giHEXw7JGWw7Ch2iMtMRQblj1SYzA29/bI4FvOucpqJdJdnXkmSMtBhMxU E6QlRhIys0yQGkMJme9NkD/fL9LnCq2VSFd35tkjLYdRRbVHWmIoVSx7pMZgqh7ZI1tN5cvh74zu TKcIOyMd3ZnnkLQcxgfVIWmJoXywHJIag/m4xiHR5yKsnZGOruQ5JC0HkSGpDklLjCRDshySGkPJ kNc4JOZccrUz0peVPIek5TAyqA5JSwwlg+WQ1BhMxjUOiTmXXC0yg69InkPSchgZVIekJYaSwXJI agwm4xqHxJxLrhaZwVckzyFpOYwMqkPSEkPJYDkkNQaTcY1DYs6lVotMzysLzyFpOYiMheqQtMRI MhaWQ1JjKBnLNftImnOp1QakB7rwBJGWw8igCiItMZQMliBSYzAZ9+uPXKUBmHN91QakT7rwlJGW w1ihKiMtMZQVljLS/DWUlUc7S46oc5hOdVUjPVPF1EQUqIkosiaixmoiiqeJKFgTUffrkIytv5tO pVQjPVTF1EUUqIsosi6ixuoiiqeLKFgXUdfoIqZTI1VID1UxdREF6iKKrIuosbqI4ukiCtZF1L0u okUwRns5CpFOsVQBLnPRTF9Eg76IJvsieqwvonm+iIZ9Ef3IFxnR/bCdkukCGM1FM60RDVojmmyN 6LHWiOZZIxq2RvRDa+TqiRK2U1VdEMVZM90RDbojmuyO6LHuiOa5Ixp2R/SjzSWPPdeQzKTmYhfR WsHDcka0pqZTaS1IP9YwPRIDeiSG7JGYsR6J4XkkBvZIzDUbTtpOpTUj/VjDdEEM6IIYsgtixrog hueCGNgFMdesJGI7ldaMdF8N0wIxoAViyBaIGWuBGJ4FYmALxFxjgdhOXTUhHVfLtEAsaIFYsgVi x1oglmeBWNgCsddYILZTUU1Ir9QyLRALWiCWbIHYsRaI5VkgFrZA7DUWiO1UUiNiMv//zL3bsiQ5 ciT43l9Rj6Q0ueLwK/xhZYTL4YhQhrND4VxE9ukI3AGvTlbeOjOruppfvzAgPMLD3bTywAxnuqs5 UxVAqMaJcAMUFzXACl0gVukCscUuEFvXBWJlLhCrdoFY5AIpSJHo+6UoRcIyy6yrxqpshRYRq7SI 2GKLiK1rEbEyi4hVW0Ts21hELLO6umhMyrPQIjIrLSJzsUVkrmsRmWUWkVltEZnfxiIyX9dV51Ez PJ2FFpFZaRGZiy0ic12LyCyziMxqi8iMzhCpse4+X5dO51EzSJ2FtpBZaQuZi20hc11byCyzhcxq W8iMbCHKnuO6TDoPmkGqExpCnNIQ4ooNIa6uIcTJDCFObQhxb2sIma/LpfOgGY86oSHEKQ0hrtgQ 4uoaQpzMEOLUhhD3NoaQ+bpcGgdWivGoExpCnNIQ4ooNIa6uIcTJDCFObQhx8DqaGqOO66Lp3GuM y4vQDbIo3SBLsRtkqesGWWRukEXtBlnON85U1pbr4unca+zKi9APsij9IEuxH2Sp6wdZZH6QRe0H Wd7mmpmZWTzdNEblRWj2WJRmj6XY7LHUNXssMrPHojZ7LMjsoYyM6/ro7DTj0VVo7FiVxo612Nix 1jV2rDJjx6o2dqzI2FFl1HFdJZ1nzah0Fdo7VqW9Yy22d6x17R2rzN6xqu0dKzzq4209hoayra7B oxmyrkIHyKp0gKzFDpC1rgNklTlAVrUDZH0LB0iMDGYJddYMVb3QAeKVDhBf7ADxdR0gXuYA8WoH iEcOEL3sxPhgFlKtZsDqhT4Qr/SB+GIfiK/rA/EyH4hX+0D88IaT3RghzEKq1TiSvdAP4pV+EF/s B/F1/SBe5gfxaj+IP/lB3uoC3hgszNrqpDEpe6ELxCtdIL7YBeLrukC8zAXi1S4Q/xYukBgZzKrq pDEpB6ELJChdIKHYBRLqukCCzAUS1C6Q8BYukBgZ19VUpzqQLAhdIEHpAgnFLpBQ1wUSZC6QoHaB hLe4SSZGxnU1dfaawWkQ+j+C0v8Riv0foa7/I8j8H0Ht/whv4f+IkcGspnrNoHQT+j82pf9jK/Z/ bHX9H5vM/7Gp/R/bW9wbQyup18hYNSPQTej72JS+j63Y97HV9X1sMt/HpvZ9bG/h+6D/MZGhGYFu Qt/HpvR9bMW+j62u72OT+T42te9jQ74PZWQwi6SL3IccexuR4yPhFJFB+LLISIh6kZHoyiODYLrI 2BnqRwazPLrIHcjx7xQ5PRJOFxmFTo+EqBoZEqcHwdSR8RZOj/g/ZlnUyR3I8e8UOT0SThcZhU6P hKgaGRKnB8HUkXG+EuYvfeFH/N91/dQpDiubo7TJosqoXCKEL40qU9MlkugkUWWULpGdgd2uo39s 1JQl/r/N28PmXf7n6gNIsfOf/lP+A64Mrw6r6+KrUxxwFr+kyFyScLqwKjSXJETVsJKYSwimDit8 j8whKCYurG5veF1YTSVhxazcKs5Fi19SZDtJOF1YFdpOEqJqWElsJwRThxWyndyDopm7ENiwsq/s rXaGV4fVddnXKU5Mm5tW5FlJOFVYtYWelYSoGVatxLNCMG1Ytdiz8uhrWsOGVYEIJoZXh9V1zdgp ruiLX1JkdUk4XVgVWl0SompYSawuBFOH1W9YXXJYmGV0Ng7sWRm8uiwzddPY3sUBr2Ej7UF5/pPu uFeH4HVx2inu84s/iMhLk3C6ECz00iRE1RCUeGkIpg7Bs5fGmnGdOsMq57Y0T5Fo++bpn+9MGjkp zZT3v+yMe20kttfFcKe4OTD+LiKjTsLpIrHQqJMQVSNRYtQhmDoSt3MkNk3bHsJwjU85qp4ZzWBN 3zJOL81SRXtdNHeKGwbnphMZehJOFUFdoaEnIWpGUCcx9BBMG0Fd/1e3ANZeF9yd4ly6+B1FZqCE 00VVoRkoIapGlcQMRDB1VOEjYW4qFkfunTsP0VZWGPHYPzG8Oqyuq/VOcahd/JIiJ1HC6cKq0EmU EFXDSuIkIpg6rPAFQzkoBjeOYYYLYDHAzHfC6s7w6rBilusVh97NTS+yISWcKqz6QhtSQtQMq15i QyKYNqz68zE0w7Auh+ByZlkOERVfrocBlC3SNmYVXnEQXvzbRdakhNNFS6E1KSGqRovEmkQwdbSc rEkUD+McmsOIaW794LuuTend63hI0Vq7+MDd7LZm7ubZt4cR0xins35x6+Zdb1zZiIlZhFfcsxi/ o8jWlHC6qCq0NSVE1aiS2JoIpo6q8/1GW5P+d5C45ul/B4W7VCxsRVFEMevvitsY52YQ2qEGpR1q KLZDDXXtUIPMDjWo7VDD+QCcv4KZHbP8rrjDMX5HoZVqUFqphmIr1VDXSjXIrFSD2ko1nKxUv7ZN 47owHRad4jiynbqpH+exbdvuoH5tQ//rwjhMzTRP/SOq6FWM13Xqh61ru64oqpgVdcW9j/E7Cm1Y g9KGNRTbsIa6NqxBZsMa1Das4WzD+suqX3ddGV8azSh9FFqwRqUFayy2YI11LVijzII1qi1Y41vc wBQj47rivTSakfYodFGNShfVWOyiGuu6qEaZi2pUu6jG8a+rr7mudtNAWxFRQgPVqDRQjcUGqrGu gWqUGahGtYFqXM5jotV3a5wLP0Jq9F3XuNU3vg3GhvWwHxyHcos3/Ra2tTXdejiiY2iG0dqu8dPk TdkhDB2z2K04hHBuJqF/alL6p6Zi/9RU1z81yfxTk9o/NX3HP2UG45uGt+Xlf763h/JgeHVYMYvd iqsp45cU+qcmpX9qKvZPTXX9U5PMPzWp/VPT2T81jJ19xNa+BTcd1gIORYIRE7PirbizMn4Bod9p UvqdpmK/01TX7zTJ/E6T2u80nf1O4zo03dIshzl/t4Zu8mEOdomPfHxEz2iaoRmDdfO8DF3/CKJf x7Ed+812a2utNc1UFFXMirfivsv4HYXepUnpXZqKvUtTXe/SJPMuTWrv0nT2Lv2Fx+HMirfinsy5 sUIvk1V6mWyxl8nW9TJZmZfJqr1M9q/Py9RdV7yXXjMOt0Ivk1V6mWyxl8nW9TJZmZfJqr1Mdj5H VRxYx7HzcXbX+M6M0+xsS6fHukdUzSGEaWtN3wXbHoPKbqazPv5f793cDH1RUF0XvJdeMwq3QieT VTqZbLGTydZ1MlmZk8mqnUw2nIPKj4Od2vUwpDKm75vRxqjyfjWHYzxda9uWAqE3dtoOF3b61U7r aDvvQ4zPMJYEVc+seXeacfos9DHNSh/TXOxjmuv6mGaZj2lW+5jms4+pjv6lT7CmHdzSu/0w0Mda whn4/JcdcK+ORGaNvdOM7WehR2pWeqTmYo/UXNcjNcs8UrPaIzVfPFJD7MuG8dC90YlcLoy9MbZp 5+UwujdR4ZwdGhvVdJzt+IjEYZzXbRu6Lgqw68a2qH9j1tlbzfh+FnqkZqVHai72SM11PVKzzCM1 qz1SM77yK3c1Q+dMPyry3x8Mrw6r60L70mqSFZzQKOWURilXbJRydY1STmaUcmqjlDsbpf6yCxH9 dY19MZo8BSc0STmlScoVm6RcXZOUk5mknNok5c4mqb/8QkR/XYZfjCZNwQlNUk5pknLFJilX1yTl ZCYppzZJub++s6r66zL84jRD9UVolFqURqml2Ci11DVKLTKj1KI2Si0no9SvfUeJbf18yH/ph25t x2aI0wnrm8PmTuvjwN4vLiyTCW6dH1G1dH2YzToZEwfq3VR0lVV/XYpfZs1QfRGarBalyWopNlkt dU1Wi8xktahNVsvZZPVX0FcxS/GzZqS+CI1Wi9JotRQbrZa6RqtFZrRa1Ear5Wy0+iuIKmYt3mpG 66vQaLUqjVZrsdFqrWu0WmVGq1VttFpPRqtfgzV7BvqugM02bPFp0wrCsBz2dyb/SFi347Ic3Xtj lEwicmXpDAOzEG81I/VV6LFalR6rtdhjtdb1WK0yj9Wq9litZ49Vu1DZMad4nuzmfRxOuXb2oXuM qUQmq4FZMp80CQur0GS1Kk1Wa7HJaq1rslplJqtVbbJazyarq7ald8e+afbryh5v1kQmOvDnfLzZ 4x/+UKkH5fNfdsC9OhKZZfZJkyCxCo1Zq9KYtRYbs9a6xqxVZsxa1cas9WzMGoY49Ors9biNNmzT +rzebpvnfy7biOvcT6OZenD+xp3yvI14xz0wp7c0rZ+88d6++iy+gVm8nzS5F17o+fJKz5cv9nz5 up4vL/N8ebXny3/f83WLDtO4tetfeSrka8/iu1He/zLpWXzDddF/VS36e6FPzCt9Yr7YJ+br+sS8 zCfm1T4xf/GJXSKROsfD5CGKdR97Lds2/do0o60zIx2uK/2raqXfC91hXukO88XuMF/XHeZl7jCv dof5izvsL77OMVxX+ldVSnQQ2sOC0h4Wiu1hoa49LMjsYUFtDwtne1hrWjuH67BubajGjc1JKe81 62VYd6uZL+6wM+V5zHbHvToQr5sDqyoDOwjdYUHpDgvF7rBQ1x0WZO6woHaHhbM7bFvDvD4tuNE5 tofwy4cm92k5jea4lfq0647Aqkq9DkJLWFBawkKxJSzUtYQFmSUsqC1h4Xxs1l+BUjI7Aosm5WMT OsI2pSNsK3aEbXUdYZvMEbapHWEbuknwPvFrl2brfsNoaF9zfG1ieG1YjZe1Cu+dRvc2oYFnUxp4 tmIDz1bXwLPJDDyb2sCzrSCsdIv/42XtIEaGQsYipygyCKeJjIgvjAxCVIwMohNERoQpI+PGUD8y mKUATfZhZBRGhs5IE/HFkVHVSEN0osjQGmluDPUjg5nOa1IITSMzwxBOFxmlZhhCVI0MkRkmwtSR ga5tU0YGM7/W5AEaIzO0EE4VGabU0EKImpFhRIaWCNNGhkEnBykjg5kua/LyjJEZUwini4xSYwoh qkaGyJgSYerIGN4mMq5T3lWTW2eMzH5COF1klNpPCFE1MkT2kwhTR4Z7k8iYrma2VZMeZ4zMDkI4 XWSU2kEIUTUyRHaQCFNHxvY2kXGxrNEiqGIE2sq8F4RTRUZb6r0gRM3IaEXeiwjTRkbbv01kXCxk 3nvNCLSVeSEIp4uMUi8EIapGhsgLEWHqyDh7IbrDGuq+qH6wiGmihVkX9ZpRaSuzOxBOFy2ldgdC VI0Wkd0hwtTRgq71UkYGsy66akalncyyQDhVZHSllgVC1IyMTmRZiDBtZHTd20TGZV00RoZmVNrJ PASE00VGqYeAEFUjQ+QhiDB1ZNi3iYzLuqj3i8KHaTqZJYBwusgotQQQompkiCwBEaaODHRKjDIy LuuiMTIUtkrTy7b1CaeKjL50W58QNSOjF23rm167rX9jqB8Zl3VR753CGml62YEthNNFRumBLYSo GhmiA1siTB0Z06sj41dTFBuXlVEfBs0YtJe5Ngini41S1wYhqsaGyLURYerYeBvXhr2sjMbI0IxB B6FrY1C6NoZi18ZQ17UxyFwbg9q1MbyNa8NeV0ZDrxmDDkLXxqB0bQzFro2hrmtjkLk2BrVrY3gb 14a9royGXjMGHYSujUHp2hiKXRtDXdfGIHNtDGrXxvA2rg17XQUNnWYMOgpdG6PStTEWuzbGuq6N UebaGNWujfFtXBv2ugoaOoVv2IxC18aodG2Mxa6Nsa5rY5S5Nka1a2N8G9eGva6Chk7jGx6Fro1R 6doYi10bY13XxihzbYxq18aIXBv5TZShPk/ToDgR+cHw6rBillCDxnQ8Ci0fo9LyMRZbPsa6lo9R ZvkY1ZaP8W0sH/a6hBq8Zvg6CS0fk9LyMRVbPqa6lo9JZvmY1JaP6W0sH/a6hBpWzfB1Elo+JqXl Yyq2fEx1LR+TzPIxqS0f0/85y4dlFlVXzZB2Elo+JqXlYyq2fEx1LR+TzPIxqS0f09tYPmZmUXXV DGmt0PJhlZYPW2z5sHUtH1Zm+bBqy4d9G8vHzCyqTppRqRVaPqzS8mGLLR+2ruXDyiwfVm35sG9j +ZiZRdVJY0S2QsuHVVo+bLHlw9a1fFiZ5cOqLR/2bSwfM7OoOmqMyLPQ8jErLR9zseVjrmv5mGWW j1lt+ZjfxvIxM4uqo8Z0PAstH7PS8jEXWz7mupaPWWb5mNWWj/n1lo+iyLguqm5GMwKdhYaPWWn4 mIsNH3Ndw8csM3zMasPH/DaGj/m6Lro1mhGoExo+nNLw4YoNH66u4cPJDB9Obfhwb2P4mK/roluj GYE6oeHDKQ0frtjw4eoaPpzM8OHUhg/3NoaPmVkX3TQjUCc0fDil4cMVGz5cXcOHkxk+nNrw4d7G 8DEza6CbZgS6CA0fi9LwsRQbPpa6ho9FZvhY1IaP5U0MH3GweY2MoLEcL0LDx6I0fCzFho+lruFj kRk+FrXhY3kTw0f8PyYyNJbjRWj4WJSGj6XY8LHUNXwsMsPHojZ8LG9yTIdpmDVQr7EcL0LPxqL0 bCzFno2lrmdjkXk2FrVnYznf2vJm+6+mua6Lbr1mVLoKfRyr0sexFvs41ro+jlXm41jVPo71TXwc 8WdmIkMzKl2FPo5V6eNYi30ca10fxyrzcaxqH8d69nG0C5Ud765zU9O3jWvXKEf9li491MQMs2La acarq9DNsSrdHGuxm2Ot6+ZYZW6OVe3mWN/EzWEaZsW004xXvdDN4ZVuDl/s5vB13Rxe5ubwajeH fxM3h2mYFdNWM171QjeHV7o5fLGbw9d1c3iZm8Or3Rz+TdwcprmumG6txmPshW4Or3Rz+GI3h6/r 5vAyN4dXuzn8m7g5THNdMd2MxmMchG6OoHRzhGI3R6jr5ggyN0dQuznC690cRcc0xOBgYkPjKA5C P0dQ+jlCsZ8j1PVzBJmfI6j9HOFN/BymPa91zH7aNCONbfStJDIirtdERsTHv/pl/eXlT25/Lvsr Qo0DQMV4/Pbrj/vDf4BPhfRcFp4idJnijz+/C1/XPY4eLM/l9Mf04CuIYmtT7+5ub7K7a9rrpa/L otj3b7aJVuEEV/pMc6u60meax+LYIlTsef7j3fuXr3/+uGbU/iqiZi6aIsjFCaD/8PPLp+Xfv4Zv B+ypkH4mAziWKXJ8ff/y+dOn46c/FdEfbjzAxzj8+tkdoPsr6hhHy6Ooa6Z4p/ddGsGhkDprti0R h+jqIm3vemOo3wKuF9QsTrOSHNZ1EenuunqV7q7e3PHf3n0I/tESzqWRpWd7ykgy5GZ0f+/L1+XQ np6K6Y+xXKQRDw0+Pnx+CrLHa3rIDvwFMvVWe+7Cm3juTHteQZqjCGpWkLbknROotwmNSr3TTKVY vU2I3/U/4lN/+fzl0xr8z19uj+W5jBgGNpQ2mdVuU1vttjex2pn2vHC0xFmuZhK4LcKAWJQBsYgC YlEHhMxHtal9VNvJR/XraI0b3bY99iXCZuZlMOvQNd1g3OHMg7Vz8+CN7cZ1mvrOPaaJ8+zbtY8j vJX2UFdXFEzntSYKJvmscW2bXhJMhNMEU8IXB1NCqYIpMZQHE8F0wbQzHDe5NtPEqHnEUnxP09ip 68MYWmPGRyzZ1fbN1lPtQBcSP2Ipvj0W0G5ZT5b/olA6L05RKMnTBuJfPwtDaVaG0iwKpVkdSpL9 UoKpQ+myX9ptMXam8RFL1oYYMtNifW+bZWoP/VLsLZq2a1YbmdpHJI128USyNtsWMWAt60v8/17+ 4L78Er5++2MKqPEWUM1wOhCsXfppnKXrFu3qtkWwbnHDidctbnirGlnfSFa0LLZc5ngZEeKUln7h Tx9efjqOo5/LIsMyAYLSgMwwTUA+GA4BSX1Rb8fhEZDG9WPXNKN98gL92tohlcQubB22QzQuc5ve PZi5W+OcjY3Gz+7H4F348Cn3bdMjFJ/7tjbOtLyVjrkiOvbec/Ek74YTT/Iy3ukmeTeSoSQUEyJG 0i8fXugX/vTzt5c/xUAIXzL4Ukw8duV5wnLkORPQF/dMLHuJTmeYLpb9RaeLZwEwKE+5vfHDzCC+ zLD1Xeusd8VBmXGrOCgTngT7tSv+N4R7iqc/hPefmXjKxZHHXVb6bzTFQZFgqqC4M7xJUAzXoBDf Y9gGM82jICgyTh4UCV8UFBmhDopMUxwUCaYKijvDmwTFyASFdA0p/amLm0RBQUujmqBYnC8MioUW yg9P/737+aP/8mekQM/VNDRqN55XFiWqRYQHAzwvMTRd56Z5hucl0j+/eV7igaEkxOw1xIx4sB5W v6w0aC4NsYRz8hBL+KAbISUSvIfNjJAyInaXP39wL1/C+t69+/AUoNdy2qS5zCJvROWBSTBdYO4M iu7rlw+HyLKPyGqYsbd0iYpGZN0siKyMk0dWxoeCzishgk2dV/5h7p1VfgkHrV35QkKGace63fyd vil2YLPrNH3TneEaP8vP2yGA5kcAGSaApAtT6WsGP4oCKBzWASQBFA5DKunkrQt4XMVP3iIixn38 dZ/C8PGa/uD2OpTKSFEgqtzcDwZFXwRjqb3G0io9AaNdgxsWWlsqjaWEG+WxlPBOGUuJBPZoXCwR gnq0+Ot+/ezWcA6oY2Hk6FomPBJHeVQRTBdVO8Nvdm+tG1RDrztDQUj2TEhKLThphGkEIZlx8pDM eGVIZpKSkEwIZUhmDtFsQLXH/GD4zmzAbtrZQGIoCMnpGpKL1BPThnmxgyQkE04RkgmvDclEUhSS hNCGZOIoD0mC6UJyZ/itkBz7xaoGgXeGgpBkVvAX8WJp8GYVhWTCKUIy4bUhmUiKQpIQ2pBMHOUh STBdSO4MvxWSzehXXUjuDK8OybFhJrZOs1TrGplwR5xKuCNeL9yuKRVu1+iF25WfnJNhWuF22pNz YFQxs12nWet1oo4u4XRRpe/oEklpVKk7usQhiiplR7czfGc4qFuAuTMUhCQzaZYbOTbXzZJJc8bJ QzLjlSGZSUpCMiGUIZk5ikMywVQheWf4zZCcOu1+xY3hGpLuZ//uWwpHyo/L8dicu8ilm4dGOhY0 neuC9740Hm+4TRqPGU95SSVOtRtqbn6XfpmnTYZjCQXEZd/rBi6NowzTxNGD4TfvCRuXEI5x9Jth cwBcw+adfx9y1JhH1JztaN3cb1FYf/ibf/zX//VD87e/Wz///PXbp88vf4gM7/cf9VRIjfzkT6G/ xbiti+PB3737/O7lYwycJ4pTIbVxByhM8zv6dTKO/otiduTfPMWfk2hX9/79A3QsoaeJsUd7Vnd6 Gul9TTMNbX9q1flhbpRb+nBnPdf4/LziL/dCz+HFrZ/f3X/MRxF9NbZVJIo7/BlJu/6XrZYbaNs7 AHrjN3/sAHJJ+kTwkaJWoUtqejD81jDy190h91uDSTbe+2u8h6jaOd6NON7pDw9hbun3Fsb7TkE7 Gd+N9/3NW1se70esKt5XGO+rPt5XSbyvqngv32TOMHW8o03mY6e+eVYFfh22yU79OA/buI7z5LAs JIb7559xr2w/A9N+hvHWflpl+3FG3X6cKWg/zsjbT8Yq2s809KD9UI2y/WSKwvZDIHn7iWhJ+4kw Zfu5MSiWHdhIH6+R7kN/i/ROGelLo470pSmI9KWRR3rGqiJ9hpE+6yN9lkT6rIr0ci9KhqkjHXlR VJE+MZE+tbdI77WRvukjfSuJ9E0R6Zs60gOM9KCP9CCJ9KCK9HKzS4apI11rdmEj3TKRThmBKdIH ZaSvQR3payiI9DhcFUd6xmoifexQpMcabaQnitJIjyBFpI/lRzFmmDbSR+1RjGykz9fVwO4+zx2V ke69OtLpFL5XR3r8OHGkZ6wq0i2MdKuPdCuJdKuK9PIEoQxTR7o2QYiL9NMRcTnS7zPSSRnpYVVH euxrXx/pYZVHesaqIt3DSPf6SPeSSPeqSBet60eYOtK1R2Wykc7s8LT3GalVRvq2qCM9n53yykjf FnmkZ6wm0qcWRXqs0UZ6oiiN9KnVRPpUfvRnhmkjfdJe5MpGestE+n1GOusifWucNtIjxesjPb5Z HOk3rCrS0a4U1agjXbArRSBNpIt2pSJMHenao/bYSL/uvw7tfUZq5BuwOXzMrA51MxeEupnloZ6x qlBHG1JUow51wYYUgTShLtqQijB1qGuzHtlQv269Dmbahy9Gufe6tVYd6q0tCPXWykM9YzWhbg0K 9VijDfVEURrq1mhC3ZbnmmSYNtSt9jxDNtSvu6SDafbxi1Fuk27dpA71bioI9W6Sh3rGqkJ9hKE+ 6kN9lIT6qAr18qM4Mkwd6t89ikNsqzHXzdKhme/DGOVu6daP6oDvx4KA70d5wGesKuAXGPCLPuAX ScAvqoAvd+lmmDrgv3vF8yngf23bKR20VhD5183TYblPVY1y93QbBnXkD0NB5A+DPPIzVhP5c4Mi P9ZoIz9RlEb+3Ggify5PxMkwbeTP2kQcNtSvu6fD8pirKrdPt7FXh/rYF4T62MtDPWNVoT7AUB/0 oT5IQn1QhXp5dlCGqUNdeyc3G+rM9un4mKsq90+3qVOH+tQVhPrUyUM9Y1Wh7mCoO32oO0moO1Wo l18ynmHqUNdeMs6Fesvsn46PuapyA3Wzakt8pCgIdSu3xN+wqlDfYKhv+lDfJKG+qUK9/Nb0DFOH +vYWoc5soA6PWapyB3Wb1e71SFEQ6rPcvX7DakLdQfe607vXncS97lTudSdzrzu1e929hXu9ZXZQ hxhct1DXbqE6tX09UhSEupPb129YVahD+7rT29edxL7uVPZ1J7OvO7V93Z3vZLBd1/adqbgS2TIb qr1b9wQl7YaqU7vZI0VJ4Mvd7DesKvChm93p3exO4mZ3Kje7k7nZndrN7t7Czd4yG6r9PXe71W6o Lmo7e6QoCPVFbme/YTWhvkA7+6K3sy8SO/uisrMvMjv7orazL9+1s8v7dmZbdbtPVVvttuqqdrVv a4GrnZKVxQG/ql3tC3S1L3pX+yJxtS8qV/sic7Uvalf78l1XuzzgmW3VcJ+wttptVa82t0eKgoD3 cnP7DasKeGhuX/Tm9kVibl9U5vZFZm5f1Ob25bvmdnnAM7up4T5tbbW7qUHtcY8UBQEf5B73G1YT 8Cv0uK96j/sq8bivKo/7KvO4r2qP+/pdj7s84Jk9Vf+Yrmr3VDe91X0rsbpvCqv7pra6r9Dqvuqt 7qvE6r6qrO6rzOq+qq3u61tY3VtmT9U/pqu6PVU3NNo9VaJ4dajTm6WhvmNVoQ6t7qve6r5KrO6r yuq+yqzuq9rqvr6F1b1j9lTXe/Z1q9tTdYPR7qkSRUGoG/Ge6o7VhLqHVnevt7p7idXdq6zuXmZ1 92qru38Lq3vH7Kmuwyg+dnTyo+jYUcJpHorMlO3VpmyvvR+RfSjM7t9yzxRudbt/bui0u39EUdD/ dOLdvx2r6n+gHdvr7dheYsf2Kju2l9mxvdqO7b9rx5aE+nW/bxzvE6hOt98Xw0e730cUJaEu3u/b sZpQD9B/HfT+6yDxXweV/zrI/NdB7b8OzXmju29t0zw2un/tKHht15n4b3tYSBitSTV0YH+k6h5P wzY2FxFgsAVN5LpPOI73iVen2yd0Q6/dJySKgibSi/cJd6yqiUDfdtD7toPEtx1Uvu0g820HtW87 DOcmknrz5nD67V7SWD/fWs/XY+vZltQgzDGDxwaXYOkNBW3kurU4DvcZW6fbWnTDoN1aJIqCNjKI txZ3rKqNQMN30Bu+g8TwHVSG7yAzfAe14Tu4cxuZrT03ELoIzJqFnNxP69FtbgPrYJ9S3Kwze6sx 1swFTeS6GTkO94O2Ot1mpBtG7WYkURQ0kVG8GbljVU0EGsWD3igeJEbxoDKKB5lRPKiN4uEtjOLd dRty7B9nP+u2Id0wabchiaIg1CfxNuSO1YT6Bo3im94ovkmM4pvKKL7JjOKb2ii+nYziv47b1C2P pxFfTpS+TP8eDzOKbaLhNRUH/3gO4zaGMRWPoS9oGtcNy7G/5zt3ug1LN1jthiVRFDQNK96w3LGq pgGN5ZveWL5JjOWbyli+yYzlm9pYvp2N5as3dvbnmzSoGfje+K5vLs+jaSJm7lp7eR6pxo6b5S/q ulOe/7A7jsF0oV8bu1ziqqE7YH3bcJhcNV0/J1UY10BMAz7HdPBzIh2PaTz8nEgHMCP8nEgHMA38 nEjHYtZtxp/TXD9nDmFKy+88huh4TFi5z9npAGbkPmen4zGejZ2dDmBm+DmRDmBa+DmRjsesK/6c FnxOnIsgTKTjMcsGPyfSAYyFnxPpAKaFnxPpeIxb8Oe04HPcAD/HMf1BqpgD/JxIBzAWfk6kAxgD PyfS8Ri74M8x4HNsDz8n0vGYKcDPiXQAM8HPiXQAY+DnRDoeMzr8OQZ8ztjDz4l0PGbw8HNGRn8y ZoKfM3j0Oaz+7HQ8pmf1Z6cDGFZ/djoe07H6s9MBDKs/Ox3AsPqz0/GYltWfnQ5gWP3Z6XiMwfrT Iv0xWH8M0p8G649B+tNg/WmQ/jRYfxqgP8OG9acB+jNsUH+IjscEqD9EBzBQf4gOYKD+EB2P8VB/ iA5goP4QHY9Zof4QHcBA/SE6gIH6Q3Q8ZoH6Q3QAA/WH6HiMg/pDdAAD9YfoAAbqD9HxmBnqD9EB DNQfouMxFuoP0QEM1B+iAxioP0THYyaoP0QHMFB/iI7HjFB/iA5goP4QHcBA/SE6HjNA/SE6gIH6 Q3Q8pof6Q3QAA/WH6HhMB/WH6AAG6g/RAQzUH6LjMS3UH6IDGKw/LdIfg/WnRfpjsP4YpD8G649B +tNg/TFIfxqsPw3Qn37D+tMA/ek3qD9EBzBQf4iOxwSoP0QHMFB/iI7HeKg/RAcwUH+IDmCg/hAd j1mh/hAdwED9IToes0D9ITqAgfpDdAAD9YfoeIyD+kN0AAP1h+h4zAz1h+gABuoP0QEM1B+i4zEW 6g/RAQzUH6LjMRPUH6IDGKg/RMdj+LXbnQ5goP4QHcBA/SE6HjNA/SE6gIH6Q3Q8pof6Q3QAA/WH 6AAG6g/R8ZgO6g/RAQzUH6LjMS3UH6IDGKw/LdKfFutPi/THYP1pkf4YrD8G6U+D9ccg/Wmw/jRI fxqsPw3Qn27D+tMA/ek2qD9Ex2MC1B+iAxioP0QHMFB/iI7HeKg/RAcwUH+IjsesUH+IDmCg/hAd wED9IToes0D9ITqAgfpDdDzGQf0hugumX2w/WthfEx3AsP31TsdjJra/3ukAhu2vdzqAYfvrnY7H jGx/vdMBDNtf73Q8ZmD7650OYNj+eqcDGLa/3ul4TM/21zsdwLD99U7HYzq2v97pAIbtr3c6gGH7 652Ox7Rsf73TAQzbX+90PMaw/fVOBzBsf73TAQzar090PKZB+/WJDmDQfn2iYzHDhvbrEx3AoP36 RAcwaL8+0fGYgPbrEx3AoP36RMdjPNtf73QAw84Xdjoes7LzhZ0OYNj5wk4HMOx8YafjMQurPzsd wED9IToe46D+EB3AQP0hOoCB+kN0PGaG+kN0AAP1h+h4jIX6Q3QAA/WH6AAG6g/R8ZgJ6g/RAQzU H6LjMSPUH6IDGKg/RAcwUH+IjscMUH+IDmCg/hAdj+mh/hAdwED9ITqAgfpDdDymg/pDdAAD9Yfo eEwL9YfoAAbrT4v0p8X60yL9MVh/WqQ/BuuPQfrTYP0xSH8arD8N0J9+w/rTAP3pN6g/RAcwUH+I jscEqD9EBzBQf4iOx3ioP0QHMFB/iA5goP4QHY9Zof4QHcBA/SE6HrNA/SE6gIH6Q3QAA/WH6HiM g/pDdAAD9YfoeMwM9YfoAAbqD9EBDNQfouMxFuoP0QEM1B+i4zET1B+iAxioP0QHMFB/iI7HjFB/ iA5goP4QHY8ZoP4QHcBA/SE6gIH6Q3Q8pof6Q3QAA/WH6HhMB/WH6AAG6g/R8ZgW6g/RAQzWnxbp T4v1p0X6Y7D+tEh/DNYfg/SnwfpjkP40WH8apD8N1p8G6E+3Yf1pgP50G9QfouMxAeoP0QEM1B+i AxioP0THYzzUH6IDGKg/RMdjVqg/RAcwUH+IDmCg/hAdj1mg/hAdwED9IToe46D+EB3AQP0hOoCB +kN0PGaG+kN0AAP1h+h4jIX6Q3QAA/WH6AAG6g/R8ZgJ6g/RAQzUH6LjMSPUH6IDGKg/RMdjBqg/ RAcwUH+IDmCg/hAdj+mh/hAdwED9IToe00H9ITqAgfpDdAAD9YfoeEwL9YfoAAbrT4v0x2D9aZH+ GKw/BumPwfpjkP40WH8M0p8G608D9KfdsP40QH/aDeoP0QEM1B+i4zEB6g/RAQzUH6LjMR7qD9EB DNQfogMYqD9Ex2NWqD9EBzBQf4iOxyxQf4gOYKD+EB3AQP0hOh7joP4QHcBA/SE6HjND/SE6gIH6 Q3Q8xkL9ITqAgfpDdAAD9YfoeMwE9YfoAAbqD9HxmBHqD9EBDNQfogMYqD9Ex2MGqD9EBzBQf4iO x/RQf4gOYKD+EB3AQP0hOh7TQf0hOoCB+kN0PKaF+kN0AIP1p0X602L9aZH+GKw/LdIfg/XHIP1p sP4YpD8N1p8G6U+D9acB+mM2rD8N0B+zQf0hOh4ToP4QHcBA/SE6gIH6Q3Q8xkP9ITqAgfpDdDxm hfpDdAAD9YfoeMwC9YfoAAbqD9EBDNQfouMxDuoP0QEM1B+i4zEz1B+iAxioP0QHMFB/iI7HWKg/ RAcwUH8MOD+kNxPUH6IDGKg/RAcwUH+IjseMUH+IDmCg/hAdjxmg/hAdwED9ITqAgfpDdDymh/pD dAAD9YfoeEwH9YfoAAbqD9EBDNQfouMxLdQfogMYrD8t0h+D9adF+mOw/hikPwbrj0H602D9MUh/ Gqw/DdCfZsP60wD9aTaoP0THYwLUH6IDGKg/RAcwUH+Ijsd4qD9EBzBQf4iOx6xQf4gOYKD+EB3A QP0hOh6zQP0hOoCB+kN0PMZB/SE6gIH6Q3QAA/WH6HjMDPWH6AAG6g/R8RgL9YfoAAbqD9EBDNQf ouMxE9QfogMYqD9Ex2NGqD9EBzBQf4gOYKD+EB2PGaD+EB3AQP0hOh7TQ/0hOoCB+kN0AAP1h+h4 TAf1h+gABuoP0fGYFuoP0QEM1p8W6Y/B+tNy+Zrej/OG+2vD5WsmDJ/feKMDGD6/8UbHYwKf33ij Axg+v/FGx2M8n994owMYPr/xRgcwfH7jjY7HrHx+440OYPj8xhsdj1n4/MYbHcDw+Y03OoDh8xtv dDzG8fmNNzqA4fMbb3Q8ZubzG290AMPnN97oAIbPb7zR8RjL5zfe6ACGz2+80fGYic9vvNEBDJ/f eKMDGD6/8UbHY0Y+v/FGBzB8fuONjscMfH7jjQ5g+Pz6Gx2P6fn8+hsdwPD59Tc6gOHz6290PKbj 8zVvdADD59ff6HhMy+fX3+gAButPi/SnxfrTIv0xWH9apD8G649B+tNg/TFIfxqsPw3SnwbrTwP0 x25YfxqgP3aD+kN0PCZA/SE6gIH6Q3QAA/WH6HiMh/pDdAAD9YfoeMwK9YfoAAbqD9EBDNQfouMx C9QfogMYqD9Ex2Mc1B+iAxioP0QHMFB/iI7HzFB/iA5goP4QHY+xUH+IDmCg/hAdj5mg/hAdwED9 ITqAgfpDdDxmhPpDdAAD9YfoeMwA9YfoAAbqD9EBDNQfouMxPdQfogMYqD9Ex2M6qD9EBzBQf4gO YKD+EB2PaaH+EB3AYP1pkf4YrD8t0h+D9ccg/TFYfwzSnwbrj0H602D9aYD+TBvWnwboz7RB/SE6 gIH6Q3Q8JkD9ITqAgfpDdDzGQ/0hOoCB+kN0AAP1h+h4zAr1h+gABuoP0fGYBeoP0QEM1B+i4zEO 6g/RAQzUH6IDGKg/RMdjZqg/RAcwUH+IjsdYqD9EBzBQf4gOYKD+EB2PmaD+EB3AQP0hOh4zQv0h OoCB+kN0AAP1h+h4zAD1h+gABuoP0fGYHuoP0QEM1B+iAxioP0R3xQz9aPizgXY6/nM6XrNudAAD NYvoeEwLNWvi7qTJGKxZLdIs/k6anY7H8HfS7HQAgzWLu5MmVfB30ux0AIM1i7uTJldgzeLupKGK kb+TZqdj9zFm/k6anY7H8HfS7HQAA/dYZu5OmlTB30mz0wEM3OOfuTtpcgXc45+5O2lSBX8nzU4H MHCPf+bupEkV/J00Ox3AwD3+mbuTJlfAPf6Zu5MmVfB30ux0AAP3+GfuTppUwd9Js9MBDNzjn7k7 aXIF3OOfuTtpUgV/J81OBzBwj3/m7qRJFfydNDsdwMA9/pm7kyZXwD3+mbuTJlXwd9LsdAAD9/hn 7k6aVMHfSbPTAQzc45+5O2lyBdzjn7k7aVIFfyfNTgcwcI9/5u6kSRX8nTQ7HcDAPf6Zu5MmV8A9 /pm7kyZV8HfS7HQAA/f4Z+5OmlTB30mz0wEM1h/uTppUwd9Js9MBDNYf7k6aXIH1h7uThiosfyfN TgcwUH8sdydNquDvpNnpAAbqj+XupMkVUH8sdydNquDvpNnpAAbqj+XupEkV/J00Ox3AQP2x3J00 uQLqj+XupEkV/J00Ox3AQP2x3J00qYK/k2anAxioP5a7kyZXQP2x3J00qYK/k2anAxioP5a7kyZV 8HfS7HQAA/XHcnfS5AqoP5a7kyZV8HfS7HQAA/XHcnfSpAr+TpqdDmCg/ljuTppcAfXHcnfSpAr+ TpqdDmCg/ljuTppUwd9Js9MBDNQfy91Jkyr4O2l2OoCB+mO5O2lyBdQfy91Jkyr4O2l2OoDB+sPd SZMq+DtpdjqAwfrD3UmTK7D+cHfSpAr+TpqdDmCw/nB30lDFxN9Js9MBDNSfibuTJldA/Zm4O2lS BX8nzU4HMFB/Ju5OmlTB30mz0wEM1J+Ju5MmV0D9mbg7aVIFfyfNTgcwUH8m7k6aVMHfSbPTAQzU n4m7kyZXQP2ZuDtpUgV/J81OBzBQfybuTppUwd9Js9MBDNSfibuTJldA/Zm4O2lSBX8nzU4HMFB/ Ju5OmlTB30mz0wEM1J+Ju5MmVfB30ux0AAP1Z+LupMkVUH8m7k6aVMHfSbPTAQzUn4m7kyZV8HfS 7HQAA/Vn4u6kyRVQfybuTppUwd9Js9MBDNSfibuTJlXwd9LsdACD9Ye7kyZXYP3h7qRJFfydNDsd wGD94e6kSRX8nTQ7HcBg/eHupMkVWH+4O2moYuTvpNnpAAbfccDdSZMq+DtpdjqAwXcccHfS5Ap8 xwF3J02q4O+k2ekABt9xwN1Jkyr4O2l2OoCB+jNyd9LkCqg/I3cnTarg76TZ6QAG6s/I3UmTKvg7 aXY6gIH6M8I7aWaoP0QHMFB/iA5goP4Q3QVjN+c7Pvdlp+MxfO7LTgcwbD+60wEM24/udDyGz33Z 6QCG7Ud3Oh7D577sdADD9qM7HcCw/ehOx2P43JedDmDYfnSn4zF87stOBzBsP7rTAQzbj+50PIbP fdnpAIbtR3c6HsPnvux0AMP2ozsdwLD96E7HY/jcl50OYNh+dKfjMXzuy04HMGw/utPxGD73ZacD GLYf3ekAhu1Hdzoew+e+7HQAw47jdzoew+e+7HQAw47jdzqAYcfxOx2P4XNfdjqAYfVnp+MxfO7L TgcwWH+43JdcgfWHy32hipbPfdnpAAbqT8vlvqQKPvdlpwMYqD8tl/uSK6D+tFzuS6rgc192OoCB +tNyuS+pgs992ekABupPy+W+5AqoPy2X+5Iq+NyXnQ5goP60XO5LquBzX3Y6gIH603K5L7kC6k/L 5b6kCj73ZacDGKg/LZf7kir43JedDmCg/rRc7kuq4HNfdjqAgfrTcrkvuQLqT8vlvqQKPvdlpwMY qD8tl/uSKvjcl50OYKD+tFzuS66A+tNyuS+poof603K5LxkD9YfoeAyf+7LTAQzUn5bLfckVUH9a LvclVfC5LzsdwGD94XJfUgWf+7LTAQzWHy73JVdg/eFyX1IFn/uy0wEM1h8u94UqDJ/7stMBDNQf w+W+5AqoP4bLfUkVfO7LTgcwUH8Ml/uSKvjcl50OYKD+GC73JVdA/TFc7kuq4HNfdjqAgfpjuNyX VMHnvux0AAP1x3C5L6mCz33Z6QAG6o/hcl9yBdQfw+W+pAo+92WnAxioP4bLfUkVfO7LTgcwUH8M l/uSK6D+GC73JVXwuS87HcBA/TFc7kuq4HNfdjqAgfpjuNyXXAH1x3C5L6mCz33Z6QAG6o/hcl9S BZ/7stMBDNQfw+W+5AqoP4bLfUkVfB7LTgcwUH8Ml8eSKvg8lp0OYLD+cHksuQLrD5fHkir4PJad DmCw/nB5LKmCz2PZ6QAG6w+Xx5IrsP5weSxU0fB5LDvdBTN7P3V8HstOx2P4PJadDmBQ7mWi4zF8 HstOBzAo9zLRAQzKvUx0PIbPY9npAAblXiY6HsPnsex0AINyLxMdwKDcy0THY/g8lp0OYFDuZaLj MXwey04HMCgfMNEBDMq9THQ8hs9j2ekABuVeJjoew+ex7HQAg3IvEx3AoNzLRMdj+DyWnQ5gUB5l ouMxfB7LTgcwKI8y0QEMyqNMdDyGz2PZ6QAG5VEmOh7D57HsdACD8igTHcCgPMpEx2P4PJadDmBQ HmWi4zF8HstOBzBYf7g8llTB57HsdACD9YfLY8kVWH+4PBaqaPk8lp0OYKD+tFweS6rg81h2OoCB +tNyeSy5AupPy+WxpAo+j2WnAxioPy2Xx5Iq+DyWnQ5goP60XB5LroD603J5LKmCz2PZ6QAG6k/L 5bGkCj6PZacDGKg/LZfHkiug/rRcHkuq4PNYdjqAgfrTcnksqYLPY9npAAbqT8vlseQKqD8tl8eS Kvg8lp0OYKD+tFweS6rg81h2OoCB+tNyeSy5AupPy+WxpAo+j2WnAxioPy2Xx5Iq+DyWnQ5goP60 XB5LquDzWHY6gIH603J5LLkC6k/L5bGkCj6PZacDGKw/XB5LquDzWHY6gMH6w+Wx5AqsP1weS6rg 81h2OoDB+sPlsVCF4fNYdjqAgfpjuDyWXAH1x3B5LKmCz2PZ6QAG6o/h8lhSBZ/HstMBDNQfw+Wx 5AqoP4bLY0kVfB7LTgcwUH8Ml8eSKvg8lp0OYKD+GC6PJVdA/TFcHkuq4PNYdjqAgfpjuDyWVMHn sex0AAP1x3B5LLkC6o/h8lhSBZ/HstMBDNQfw+WxpAo+j2WnAxioP4bLY0kVfB7LTgcwUH8Ml8eS K6D+GC6PJVXweSw7HcBA/TFcHkuq4PNYdjqAgfpjuDyWXAH1x3B5LKmCz2PZ6QAG6o/h8lhSBZ/H stMBDNYfLo8lV2D94fJYUgWfx7LTAQzWHy6PJVXweSw7HcBg/eHyWHIF1h8uj4UqGj6PZacDGKg/ DZfHkir4PJadDmCg/jRcHkuugPrTcHksqYLPY9npAAbqT8PlsaQKPo9lpwMYqD8Nl8eSK6D+NFwe S6rg81h2OoCB+tNweSypgs9j2ekABupPw+WxpAo+j2WnAxioPw2Xx5IroP40XB5LqrBQfxoujyVj oP4QHY+ZoP4QHcBA/SG6C4Y2+xw8PzHRAQzcr3fc+Ympoof79UQHMHC/nuh4TAf364kOYOB+PdEB DNyvJzoe08L9eqIDGLhfT3Q8xsD9eqIDGLhfT3QAA/friY7HNHC/nugABu7XEx2LmTe4X090AAP3 64kOYOB+PdHxmAD364kOYOB+PdHxGA/9YkQHMNAvRnQ8ZoV+MaIDGOgXIzqAgX4xouMxC/SLER3A QL8Y0fEYB/1iRAcw0C9GdAAD/WJEx2Nm6BcjOoCBfjGi4zEW+sWIDmCgX4zoAAbqD9HxmAnqD9EB DNQfouMxI9QfogMYqD9EBzBQf4iOxwxQf4gOYKD+EB2P6aH+EB3AQP0hOoCB+kN0PKaD+kN0AAP1 h+h4TAv1h+gAButPi/SnxfrTIv0xWH9apD8G649B+tNg/TFIfxqsPw3QH7th/WmA/tgN6g/RAQzU H6LjMQHqD9EBDNQfouMxHuoP0QEM1B+iAxioP0THY1aoP0QHMFB/iI7HLFB/iA5goP4QHcBA/SE6 HuOg/hAdwED9IToeM0P9ITqAgfpDdAAD9YfoeIyF+kN0AAP1h+h4zAT1h+gABuoP0QEM1B+i4zEj 1B+iAxioP0THYwaoP0QHMFB/iA5goP4QHY/pof4QHcBA/SE6HtNB/SE6gIH6Q3Q8poX6Q3QAg/Wn RfrTYv1pkf4YrD8t0h+D9ccg/Wmw/hikPw3WnwbpT4P1pwH6M21YfxqgP9MG9YfoeEyA+kN0AAP1 h+gABuoP0fEYD/WH6AAG6g/R8ZgV6g/RAQzUH6IDGKg/RMdjFqg/RAcwUH+Ijsc4qD9EBzBQf4gO YKD+EB2PmaH+EB3AQP0hOh5jof4QHcBA/SE6gIH6Q3Q8ZoL6Q3QAA/WH6HjMGPXnd//zi1vfffzx h/XThw/uo//hnX8ffvj8zv9gzA/f6F9N03TzD9/8A90uHV3Q0DQ//M0//uv/+qEb//Z36+efv377 9PnlD5HiffiS/45T4e+bX9vp/JeYzg3zTLODd5/fvXz88O6Z4lQYKTrHU8Qv9btvX9znjKP/im8e Rv7NW+yliXZ1798/QMeSCL78ag/s3//936ePiHWm+7sfvrz7/MP/vb8v/mPjjzYNbR9rvh5qEsO0 Bb82sWbhanzzQ+Smn/OFHsSLWz+/u/+YjyL6aj3z52WKO/wZGUGO+z0IFH//r+sfgk9v/OYz8FiS PhF8JHmxt09ffnoJv777lqH3l/Q7tjyMUrzS++Iv+eHzp/fvPoYD+FEYKQJmeDyK+3N4/OyxuTTn 3/rxT0i/9ataQN9cWsCwTf3eAiZlC3BW3QKcLWgBzspbQMaKW0AYbdewLSDXqFrATlHUAjJI2gIS urwFJJiqBdwZDo9iGtbWPJ5GfDnGqIhPYh26x6OIL+2Wi214PIdpWJY+FS/zWtA0zFUc3DDuTcMq m8YyqZvGMhU0jWWSN42MVTWNATaNQd80BknTGFRNY5A1jUHdNIZT0xiHrTk0jfjSUPTSvw+CQS/9 rXh5PIdxCH5MxcH3BU2jvTaNOdxVY1Y2jXVUN43UP7y2aayjvGlkrKppONg0nL5pOEnTcKqm4WRN w6mbhjs3DRPsdmgaJqw0bor/XvyhaZgQH2EqDtuhaUTKKRePQ0HT6JimMbW3ptE3yqbhB3XT8ENB 0/CDvGlkrKppbLBpbPqmsUmaxqZqGpusaWzqprGdmsZgW+8fT2OwXedi9NK/58ejiC+XKRcvw+M5 RDAleDX07xLV6JmmcZ9t90bZNEKvbhqhL2gaoZc3jYzVNI2+R00j1mibRqIobRp9r2kafS9qGn2v bRqZ4dg0TJiWQ9OIqkHTbfr3dGgaUTWGXLx1h6ZhUgxRcT8VNI3h2jTsfRret8qmsXXqprF1BU1j 6+RNI2NVTWOGTWPWN41Z0jRmVdOYZU1jVjeN+f/gQtTItIDmPm7qNC0gjK5pdS0gUby2BaQ3C1vA HatqAfxSbK5Rt4DipdgM0rQAwVJsgqlbwGkp9tfW9dP8eBrxpaUhDP3bPh5FfOlMLnaH5xBfDi4X D3NB05iuTYMu6ro1jV7ZNIxRNw1jCpqGMfKmkbGapjF0qGnEGm3TSBSlTWPoNE1j6ERNY+i0TSMz HJtG1y+H2XbbDf1Cwd4N3eFRxJdpMZb+fWwaHd0/TsX9XDKlsEzTiMF4axqDsmm0jbpptE1B02gb edPIWFXTsLBpWH3TsJKmYVVNw8qahlU3DXtqGqbr/Ph4GqbrKdbjcDdOPh6PIr6k++epeGofzyGC J5OKu6kpaBrXve2pfcy2VXvbFG6bvmlsJU1jUzSNTd00PGwaXt80vKRpeFXT8LKm4dVNw39vSnH/ lV8f6sN1E3v0j9mzahM7hk8X1KHehYJQ74I81DNWE+pji0I91mhDPVGUhvrYakJ9bEWhPrbaUM8M mlD/+O1LDvV2D3UzXLYXXO8pt+Y1f+krP+IyE3f9Evr9CXz907tv6x+OTyCX0BPwl+CP6HWaCB2D 9wi9v4w4M3UAN8Z29+0PX4LzL+8/fbq1g2NJRM8zAJc+9gzTPPYHw6EFblt/1P6Gcgjizxz/fZgx mmZJm1Dx38dNqM2O9OYm/tsWBMllVOzSULxekLTDZU7q5naSB0n8zURBQjhxkGwCg0uGKYNkOxtc avQNvbk89mVa6Vet9th7cxlUxo+Iw0rhY+874ySPPeGkjz2Byx87wXSPfWeo/Njby+gnPpOqrb1v L1aopTfy1t53bS977BEnf+xt+RZThmkfe3veYmrXprONezz/X/uha7dxWxYbVcDZwwp7H3vbZNQ0 tlkHc9hrWhuzNZNt7OQaunCtIGQu/p34PCnDo2LIXAYqS9/QIW/ikAnCkAmakClfeM4wdcicF559 IJuQD4+Q2UsONV9hzcLXFITMZfM+Pk+6JKxiyFzGFIul4BeHTN/KQibi5CHTl883MkwbMv1pvvHr Qs7ptjlsV4xj45thsGZsht4142E7ew7TsjVN15lmMIcRqBuarhlb2tGem34u6WSY4cg219UlZjiy UVCKI2YVRsyqiZhVFjGrOmLWv6pOJkblc8j0TegGL32eET3Yofx53nCy53kDlz7PDNM8zwfD4Xmu h+0Ys7ptNfPWr+6wG2PWpTO0h9Tbw2bMvXQE7f3H8OlDfnjd3hi75jolXHrF3KBJ4VPeGAlHIfM+ hM9/fP7YYxHhm8uKUcbT5Qgv6d0ZmP+bFgj9ZYXqhoh/6Y8vX37++BJ+CR+/fc24YwmtDQb0eYL2 H2HK9n9jUExH2DC4Lj6kDT95GGyUQysIg+TvVITBRndjF4XBRqdz//jy7tNL+mo/vw8vP3/eQ+G5 lMJhBSSBSH7+/PL5y6d1+eT/vDMciujnQnBRMKkspg+G2sF0EXi3dIqJZ7P1smDael0wbX1pMEXE JZj8pz995MKJyiNTz0ZEJEoBRW+6hNRTIQyqTTRzjjBtUG29Mqi+hj/+HD6u4csPcX6cgqu/jx6v grW0Da2lC4OrjUMeSXARThNcEZ+Ca/3l5U9ufzr7K0KNA0BN8VPDH7NA5ZHN7XOfC+kBc3FBFIK4 iDBlXNwYDiOdLizb2gji488pOFJkDHtkjJckjn4aV2m3041DtzV0D1JZZNxwkzQybvjljv/27kPw jwg5l1IHsgCSLYfX/b0vX5dDnD0V0x9zPRAk89Cg+vaDxd/96QeMryNynC6P+4YsjbMM08TZg4Hp f+5vmrc5NgJ/2MrZTv88R+DlLIM7wzU4f/n4/svPOTbHR2yet+coNqWrJOk7bnQHhyA2N9+pYnM7 xLY4NtPgkNfVxfOIEJti+mmT8GXU4zVN666HktyQoihUDa0eDH+hefoH91M2XIzDMNzCsB+m+Xk9 sLemt8swCqfq3ey22RWH4Q0nDsMb/hGGq4vvffn67seP7v3XJ5qnmmQVuYzabmzrnS13ie9+fCLa CynOAINHEb1eVDgj6NKZn8KXj4l8zKjHa2o81vBIOvb+65+/pjf2t79zfxlx06UJZRi5WdwHP/ax 4/5KnpQMfSqin/x64kzGpxbhvn7b3/tC0fbpNrZlKn5PW+GYiprGDfDD30x/98N/+RLC//M//vMP //Qv/2Xs/+6H9G3+9mhN7U2/+dY4d1wLodLQ0OjiuBayl0LznXv3yZvUQob7SlY/Xley7CRuHk0Y zEQnRJevZEWcFTePjF9VvXQmoYB5bS99Q8SBePxpX7wL98h4vKbpswE/lGj1zaiMHw+G46EVzbCN YRwO6+9TfD1P/dSPcxxQHnb5WhunHtTZb107DuN6CMCGMsEiaor/FfE4CNschN09CKf2GoSzdA+O vmLTyYIw4lRB2HQVgrDpSoMwIoRB2JS78jNMG4TN1ZU/dNM6HRxINhcth92fW1EYH2F3K2pGHG9d jrf+EW/dNd6sdEJN36ZrZPEWcap4Swf5aOOta0rjLSKE8SYwMmWYNt4uJ/WYxi5NYw655Yt3g9+2 cZhWN7pj2PXDOvjO9n6JwwZ76PPS3CCM3bK5bV0nkEhLMdjnGBweMdhfY7DXbCHFZyWKwYhTxWDE 62OwXUpjMCKEMRiRkhhsF20MZoZjBnfT3NJSd9NlslZOg3Xpvw5HRu0lnW87Y5r2e8tFf7hNhUZa maaQm9qoqc8hF38fKzbFRbQfJ8lUKOEUU6GErzYVSmyqqVBiKJoKEUI2FSKkYCpEMM1UiPCVpkI7 VeFUaGq3vp3m5WkmRIWxX4/T/cNEaC8cDWgb/7F93RtHd1sn6Kxpzo3D0m290v64m5d2mYuXUm84 8VLqDb+ULbJnFDXlb7/+eAt2Wm72L+8+fL6FCVdD/dy1N89kw4WM5YkUiCGKWXxSv2xfX74F9+Wx o3QqpDbWgB+SztSPb39592l9+fLp/fvFrT/dSZ5K6YdpN54lDhOIxYdfCPLtOfCZCloeNlcJyVzr M9eVJP0lIKCWqFvxjflv38GHgoj14IPpWuFfPh5x+6sIWhFoefq8l+3yiS8b/b1Xtctwf+viDp/6 eE0PznJrUxGZFmz//PQ17y/pAwfwB6uWe5abwldY7lkeUn/v44b+0sml73Po5IzbnPfBdU+9XCrt 6Wyj9tDNPUrd98YAU3Ob+vTxiz8f4NJP4zhN4qmPCZNvBGOAjJOPATK+1hggs2nGAJmhZAyQEKIx QEKWjwESTNE+Er5O+7hTFY4BjLWTnZbGPjcPKh0HF7v7Y/PYS9cBNI+P7z7+u9tbSBwr5BYyWnNd jDJGMTHrh94V77bfcOLd9oxfTKUWktlGRQvJDNNjYPKAH0tS537V4Yze5STOst6//LT5r4cWcy9L 9pJre0sM63hgOIF/D1ySEZa2pmPr+vyA3V9SY7tswmWYorElfJ3Gdqc6idFwaW7p+/ztcf2tH7fY iNxx/a2JTWob7HH9LRUtoJWtv//93sa6265D3w+2P89Ew+qk3ruInlc6kKB8Jkq4VjETJfz4ertU RvTxL/3y4evLl/ef1p9eNnccl17LaUx3DbBM1Ofh7Raf4/LVv3wJ63v37sNjhHuqSIspgCq2zP/9 3//15d/+6R//5R/++b+9/MO//u9McyqkVRVm/pAo5jjC/PHTx7CPL9N/E8Bxk+aIoKuB8n42/aGH ycapkDgmbjJMHDaNa+OPFUfzf4hTgvvg9lFEPyF4+kMc+fwYvn0Mf/rl4ycf4i/1NXz55basw1RE quuFFjeqPLR/+Y8IezwCekXhw8wtCDROGeTffQkfv5Fd+6fdiHkpTpM3bnJPPOudh978xEAF1E2h P2HL0PNn3z+0u17pk5HT+hx/q6O2eubhapODhielbR+aXqS3P/1R59Lfc3n4N5I4/cxv3T7nN798 /BR/yS83PyFfR+2jA4FCc+pnFMdEHdQK2oeLofYxlt/OIUj/SY/0elFOfv+SWsfLp8/h48v6JTwC +15Cn9aAT1vtTero3e7bQexyQZp9gUdAJr/t3XtaRX350xf3+XP48kTD1yXp5glVS08RX2vp6Ub1 pIT9PF+UMH+hJyn08zzZ5kkKp9ji1+ZJClORfYUUdrsUtvN4GW52g3hRNg46Ri+QwoyTS2HGl0hh QtSQwkxURQozlUoKM0WJFCaEUgozh1gKM7yKFGaqQilMoApSmHlEUpihEilMyNpSmEi1UphJakph ZtRIYWIokML0frEUJrRQChO2phQmQoUUJnwdKbxTSaQwTFs4S+E6jm59lsJYtPSvkMLhvjY5t5ct mDj7XKVSuGzdIJHChFNIYcIXSSEhqkhhIqojhYlKJ4WJokgKCaGVwsQhl8IEryOFiapUCglUQwoT j0wKE1QkhYSsLoVEqpbCRFJVChOjSgqJoUQK6f1yKSS0VAoJW1UKiVAjhYSvJIU7lWhWODlzlsKw dfNpVkhFr5kVTrs7zJ7Or0oHDMzS5BnacGk6gRRmnFwKM75EChOihhRmoipSmKlUUpgpSqQwIZRS mDnEUpjhVaQwUxVKYQJVkMLMI5LCDJVIYULWlsJEqpXCTFJTCjOjRgoTQ4EUpveLpTChhVKYsDWl MBHq9uVvbvwa+/IPW36pFK6Tc2cpbMzYTs9SSEXbK6TQ3qXQTue9wn7z8gQ+M9JuqGCvkHCavULC l+0VRkSdvUIiqrRXSFTKvUKiKNsrjAj1XiFxKPYKCV5pr5CoivcKI6jKXiHxCPcKCSrbK4zI+nuF kVS/V0gkdfcKiVG3VxgZivYK4/sVe4URLd4rjNi6e4WRULVXGPG19gpvVKJZYTNd9grnvh/CsxTG ohGdW3aUwvnuUbf2LIXksZXaZqbJta1ACjNOLoUZXyKFCVFDCjNRFSnMVCopzBQlUpgQSinMHGIp zPAqUpipCqUwgSpIYeYRSWGGSqQwIWtLYSLVSmEmqSmFmVEjhYmhQArT+8VSmNBCKUzYmlKYCBVS mPB1pPBOJZDCLk5t7cU2s3TmtFcYi9pX7BX2N5f2ZOzlBFbfbI305CuSeyOxzWScalZoymwzCVFp Vmgq2WYylXZWaMpsMwmhnxUahW0mw2vNCk2xbSaB6swKjdA2k6HCWaGpbptJpBVmhaaubSYzKmeF psQ2k96vmRUaqW0mYSvPCo3GNpPw1WaFRmyb8b7dTlI4bvMyL09SSEWue4UU7idJpKsSn6SwbZZV fMVR/I7W9iIpJJxGCglfJoURUUcKiaiSFBKVUgqJokwKI0IthcShkEKCV5JCoiqWwgiqIoXEI5RC gsqkMCLrS2Ek1UshkdSVQmLUSWFkKJLC+H6FFEa0WAojtq4URkKVFEZ8LSm8UYn2CtetP88K2250 9nlWGIsWdFvUQwrteEumMHY+XSjY98u0WXkyxRLsOJUfqpRx8kOVMn7PR3//0wH5Pp1U0XDmsIix 8ZuS3n348UtUThK+Ryd+Lf99Or8KMFEue2we9OabFN9e0W8FvvLckWiFby/bu4/vvv5hV657QYT2 nFORoOPeI1z7g0ff2XDWNEKvBd3cdE0OTiR0jMEvp677l2PX3Q/g40Fn1AbOnRrSTZnf74xMB9Bi D1/E1vXwRUKVhy/ia3n4blQiO/swdufOyM9mbJ87IypCJ218/unHv//6zX17t976pLm5HzY4m+G8 aTN0Qexqj1MQv4hO3SKc5tQtwtc7dYvYdKduEUPZqVsRITx1KyIlp25FmG7i6mudSLNTFZ644YZu 3Z6ahqErIf1T06CiHh22dW0a+60p3WzMKfexT2ffKBI+GjuVN42MkzeNjJ/KEj4aa02VhI9INFRK +IhUVpnwESlcWcJHY2ejTfho6LRgecJHhC+VEj4ilc9PI8q4+xYeDyG/TgFxPbwqIZ09zQBPDM/l 1NBZp34kCvkp/uO//dM//M9/en6Ij7L0SHiCVw1Q+oYbSEW0PMmgsZWTDBqrSzJobLUkg0xVPkCJ XbCbTl3wEkZ36oKXEKc5v3kOWGfNLdOu689nHMXZSpznKJLO54HGBuVJ5xE3a5LOI95XGpMktuWx gHm4LOY0MHmqob+NadCJbLzcRvPuR3gjTa76PXsrzY1vPwDpa3gf1mPDygXUK/Ept/OQzuiN45Uj 9PGaHiybATursnQIXy1hdb5m6czdpf3k7/M0wI/hPs398ylGoY2D/udTjGKRRQN8H37xqRH13f3U +mGczjk67TapTnAeBY0o4+SNKONrNaLEVqsRZbJ6jSjzyRpRwkoaUQKqjgKL+FpHgd2oihvROMe5 77A9N6NYOC/j6Pr50JBuheuI9rA+bz98/vnLj+HWnOZ7c2Lu1LXJjyNtTtNIB5uXN6eIc5rmNI2H a12lB6JPI77blT8QPSJieH7eXtLP+3K8Ju9USCHOtrZIITkZfRq1J6NP97BkLgRe+m3rh9iNh+Ml OFTaOrMNRyvRvXRcYfi9//TjrS9/XINjutOt8atv/Cy/Bmdth9YV3wl8w4nvBM74Q++r7MsT26RY pMkMtmCRJiFW+gbLodEst6ZimaaSAHEG/PVTnN6Gd7+Elx/Dx/Dl3X5r37k4LcwCHt8eeE74tAAM YH3zO/+JpiaPJvd4TX83+HH9nPXkcKHl7dXv2UOZE0ijJYSvpCU71ZOWXKWEvszTEa5h3ug2Trs+ HeEaS9c4iY9/9PEI170UueSfGrN9DMyGi5Ikm4T4BNdtdpvkBNdtPpzAKjjBNeKHeie4krOk1sCM yAJq147r4yOCxHz5vD2i/faCPoRZTskYvx9Bnu6zfzqBPJeQpoAPFDXMBCxtmATSnfda6665O1XV hjk0XMMc0CDvfvT4PNvdsjudjx7vTNRu+UaIDf0g2QhJOMVGSMJX2whJbKqNkMRQtBFCCNlGCCEF GyEE02yEEL7SRshOVbgRklcRns4dz0sGQ3tZRRg8aBWPG0rHptkvyjXj9TayNrRi04JppxAkR/In nOJI/oSvdiR/YlMdyZ8Yio7kJ4TsSH5CCo7kJ5gm9Z/wlVL/d6rSI/nD0nSj756bRiodvF+W40nh j9LvHgAwNt1tA93M7chdWyUezvXBhF5i6kk4hakn4QtNPYSpY+pJTKWmHgIJTT0JKjb1JLTW1EMk MlNPQhaYeuj9clMPocVnVEZs3TMqI6FqyT/iay3536hEph5HZwCezqicvDmdRkJFqDPax7Bjc/fa GzsY5uZQ8RjWhGmW3Caecarrc+Z6t4lnNuX1OXPZbeIJIb0+ZxbcJp5gumN65kozvDtVrTFsP13H sGgz+d/Dr2HdG8a+gDo1bdNfnbeTOAklTmLtKEhCyTh5EkrGlyShJESNJJRMVCUJJVOpklAyRUkS SkIok1AyhzgJJcOrJKFkqsIklASqkISSeURJKBkqSUJJyNpJKIlUm4SSSWomoWRGTRJKYihIQknv FyehJLRwiJiwaIiIB4j88DCR6dY5a3m+71TP91qxg8PjhHUN29Z6O0/H3cRU2pjFxyHeYcK6l7ZI DZf3sT9//+7rt7wJMbTmsaN4vtfDN1Z+Vg9tt6wCb2vGyUeKGV/ibU2IGt7WTFTF25qpVN7WTFHi bU0Ipbc1c4i9rRlexduaqXyhIBJot7XSkoF339wDuJeQdvCbnSttZv/Hu/cv7/+0pN5g3xJ+LqNf j7m9NzNMD4Y/fXn3Lf7eX7/+HJ5pDhU0bWC2kxLVkqnyhx4e5qmQ5jg9eBp+H+HR/ceHMKaXtCEF voXfO/8D7vE6+Z1Ap6HcLl7rbRevV+vRtbtO3+fQX3fbPM2xiTfT0XqUSl3fz+txw/hRiu78jM3F vf/7nz8un37+uHfZ90yd0ZwdSIvfjHhyHwVqtCJDH+E0hj7C1zP0RbZ6hj4iq2noIz6poS9iZYa+ CNSNgcZKtvI7VbGhr222eevj8ObJ0UelYVvWZjiuCDxK0SWfH77Fj9sXBcbbKfZ903WXdNxhasWX TffB+I1Ww8tX7iOu16zcRzx0MbF3m0XE3Na424yIxjp3mxHVrLvbjCiWorvNIsK1yrvNiMPJ7zYj +Po0BrqMfdJPxrn4I5Zmbd8+fI6/c/6oX24R8FyWfC/gAZID7vhumm9dOaiUWHp29yaybPnZ/bf/ +p//+d+en9y9iH5CdjPGb6u5dZMffvLvvjzNH28l9OGG3WPwm25LwFc6lfVO9dTfxanWpcNLX+nY 361jv5p1oEg69HexdAlD4/1Tf7eXhvm7WwO7vaXvh/MKKB3OKneddWbtZHm+hNPk+RK+Xp4vseny fImhLM83IoR5vhEpyfONMFWeb8TXyvO9URVuDSxxoj046542B6iwn7tuPa6J3AuRwXrddwdMs9ur +8ac24b1sxPnm3WzaydJ20g4RdtI+GptI7Gp2kZiKGobhJC1DUIK2gbBNG2D8JXaxk5VvG3m/die ts2mPob36TzvWNShhcIfv4TP93ZxOBvies3T1IhzXjozxnF6+Ywz4+QzzoyvNeNMbGur0gxi6MrO hoiI+A0+v/scDq7f+0tqFcyyUMJJXMYZWOgyTiDdgRKh3vn44ToS+67LuI0PvF/88OwyTqVuaKdh PA7D7qVo2rkeDJXG3Jfep669Giq9eNpp+nbzbik3aSScl5s0CH/IH1CaNBLbqDFpJIYpL948w48l aeOIuUkroV+xVMPk5iSsZKkmATWmDcJXMm3sVIIEZre07pTAbDo/duPzGQBUhBKY7zMWs29R9WTP O58CsHatOIE56mvfyEZlEacalUV8xVFZZFOOyiJD4aisb6Sjsr4Rjcpip6galfVNtVFZpvpLGPLf +//r/Xt/bxm3+UokHs4HE9mm3RQ7AdMaxiDYCYi4qVHsBBC+L3IzESLUcDNFIlqrreFmIqpB52Yi iqnIzUSIoHQzRQ5avJW6mQhuRQuXGbsqFi4TQ2yauoXLzLLsLMcTjY4l9AOCZ6A5yygRiM8ySmix 6SZia/qyE6Fq0ynia2063agkvmw7r+PJl90PYTsdgk5F8BB0ttNu49ShPw37F1qUC/Jz0H3ol/Ld poyT7zZlvK02mSY2r5pMEwPM9WUn0z4nNPz84duvfzzCDgXUYhtuihuxvqEJ9cvjAIDbi/Qdubko YWh/7oU+ILaj/I1+fvfx28vnL+9+ufc5v/EGigLwMPaZxw5+DLH2EhplcYsDXpVLkfC1puuey6UY +kub3b/SU6sdTd/5ZT222hihQ3i+5jwVlbfaS/Zj18yKbRO7raJWSzhNqyV8vVZLbLpWSwxlrTYi xK3W0upZYaslzBu1WqIWt9oI1p2xvlUyTd2pqrXatVvG8LxskIrQRidutacNHdrqXMVLbNO0LJOg 1WacvNVmfK1Wm9k0rTYzlLTahBC22oQtbLUZ8yatNlMLW20Cq+7Li/ha9+XdqKq12mBWf2q1qai8 1Z5O4JjG0XbybVizDkGktYTTaC3h62ktsem0lhjKtDYixFobscVaS5g30lqiFmttBOucE0OlWe2d qlarjc/OLs+tNhehVnvczNpNyf1k+uZ8RroZZ3FiZZzJTatgkT7jNGekE77WIn1m0yzSZ4aSRfqE EC3SJ2T5In2C6e7XmCoNRO9UhYv0Yz82fdvbp2V6Kuy2sRuO1v174SvOzjHdLR+/i39Zf1G0OAyQ H2RNlywKWkfCKVpHwldrHYlN1ToSQ1HrIISsdRBS0DoIpnSlunquVCdoHW0M167vtuapeaRSM7TN 9OzBv5VadNHGw3fX3Y6W6iZzvnumH9e5EU/TxmmZjCAJMePkzSPjS5IQE6JGEmImqpKEmKlUSYiZ oiQJMSGUSYiZQ5yEmOFVkhAzleSCjYSsccFGJlJsSiUC8aZUQgs3pRK25qZUIlT0wQlfpw++U4lu 5h38crmkvpu3/nRJfSxCrueHv6ab7ocFzc1labvVLG2Pg+Rgv4xT3fw11DvYL7Mpb/4ayg72Swjp zV+D4GC/BNMZNevNa0dmXvv9wUk3OW9DH7qnwUkqXYeh3Y6XZ+ylXYN2f1I6zr2FzPupQdN5yzYO cXr5BWCdDZ0RJAhmnGJBKuFLEgQTokaCYCaqkiCYqVQJgpmiJEEwIZQJgplDnCCY4bIEwYRVJQhm Bm2CYGYRJwgmuDhBMKF1x/t2lYYBd6riBMHYy8/90Nin/EAqbKfQTcfebi+03x8L9DdHoZns2Wvb ex9on1t845YoAyrjNAsVTcUMqMymW6hoCjOgEkK4UNFIMqASTHeJVq0MqDtV4Vhg3dqxn/3wNBSg wjYY64/3r+yFYXrFMl5/W+TuYqd+Pj0wto5Rc1DANMqW8SJOtYw3jTWX8SKbchlvGkuX8aZRuow3 jaJlvEl1iTThqy3jTdcDxV5xR65t5mUY/VProELXTGN/zJ29F6KEpodyTHuWxmDOOYJ0V2MnzhHs l20yog0gwqkuyY34ehtAxKbbACKGsg2giBBuAEWkZANoUy2vJHy1+0uZ5ZVXHA/fbN745Xw8PJUu vfPTdjxtby+FtzAemsdtYNXHoVV/SWJqevn+aGi2WZCqkXCKVI2ML0nVyIgKqRqJqE6qRqZSpWpk ipJUjYzQpWokDnmqRobLUjUyVpOqkRjUqRqZRZiqkcGaa6eJQH7tNKHFR+hHbN0j9BvVXVAJX23A z9wF9apV8djJ2bY35PU79txU2rZucscZ8b10QcP+x8UeZur2C6HMeX28M4uZxSMbE8bRCrruhFN0 3Rlf0nVnRIWuOxHV6bozlarrzhQlXXdG6LruxCHvujNc1nVnrKbrTgzqrjuzCLvuDFZ03YlA3HUn tLDrTtiaXXciVF3yMNY62vFOJbr9ZFzW84amb4N7vvGciuJz+95Ym3rpNNYeu0uHbaM8iG/J7czq JVPRjFMdcebrTUUzm/KIM182FU0I6RFnXjAVTTCdUdfXWuLfqSouYk6hvy5iTgHeDPTnr/d7Z8fx fjnQ2F73+7tWbEZs6LowwbkzGSc/dybhq507k9k0585kBum5MxktO3cmYYVHBKusuwlf7Yhgxrr7 qiOCnZvGLg6on48IplLjhi083bueS+mqLJiTsn76uL378dZo2ma3yXT9cF78n9ZVczjg0s8Ck2LG aY6hIXyJSTEhapgUM1EVk2KmUpkUM0WJSTEhlCbFzCE2KWZ4FZNippKYFBOyhkkxEynG9IlAPKZP aOGYPmFrjukToeqso4ivddbRjap8TN/Z3o1t/E2fhvWp1DRNb49e8XupRw6Fox2r7R7HUZ4PzutM v81yOxZ5viV2rITT5AcSvsiORYgqdqxEVMeOlah0dqxEUWTHIoTWjpU45HasBBfasQirs2MlBrUd K7HI7VgEl9uxCK2aq0V8rbnajarYjjVtITTj4J7sWLHQh3VY1+PBu3thgEfvvvu2d3VDd/dmT9eL XAcv9mOZMOWzOssvco041UWu+UzOKnO1zKa8yDWf8llykevm10mUCp2xW1EqdML46ZwKfftLn8p+ z16KnQmECc8JrLtFttqBpDuVLOG5t4s5HVNgum20y+lM0m2c+1c3TDqjYLguoozyhhmHXHSsj2Q2 GHHyi2QzvuQi2YQYKlwkm4mqXCSbqVQXyWaKkotkE2LUXSSbOcQXyWZ4lYtkM1XhRbIJNOkvks08 ootkM1RykWxC2soXySZSp7xINpMMze8+/hw7xyP+UEARAfoRWlT95fRz/HL8OXpmLTAhaQGUuSm2 DSB0SYTz+uQ39+1p7JcLkjyAn97fVve3I/ZQkISRh+qmyWutS0x2quebXYeWuSwwfaPnkzi6dh6e jwV23dyE52Q+KoLJfB+/fc6L+9Nu4G/MYM+XA4bY8YgtyvHvWUX3uSac4j7XhC+6z5UQVe5zTUR1 7nNNVLr7XBNF0X2uhNDe55o45Pe5Jnid+1wTlWSVMiFrrFJmItX5vpFAcb5vRMsv1V7Xipdqr8rd orXebtF63S367qXandumsI3+6U5tKlxmb8fjgez3whH0vV/o/tfc+U73zndszrN138zy7Kn4LYOT 7awGp9tZDa7mzmpkU+6sBqfZWY1o8c5qcMKd1aA59iXhq7WVcD325XU3ekxD6083ejS2d836fHdB LDLQoPP1D7dBirX3dkKHD5wGKcZp2kmEi9pJZ3TthKyg9dpJZFO2k85o2klE7+3k86cYVj9t/uuh qdzLSKmYNbLEsI4HhhOY/mrwZ/slN7LPD9j9Jc382JGBKp034as1MSadd+ivgpS+z9G8sDR2CFNr 7Xg0L1DptA4+HNOyHqXIvHA3w3Xtra2Ns22HiyYFxe05ZkxmC8HpHrHVy93LGV/iXs6ICu7lRFTH vZypVO7lTFHiXs4InXs5ccjdyxkucy9nrMa9nBjU7uXMInQvZ7DG6UAEcqcDoaVOB8JWdToQoe7U mbGrdupMphI4Hfxih7Vx87PTgUpd73vzdPDMXtrCxJP37uOP9+STrp3vl59dU87D3GuST7Ze4GXO OM0GIOHrbQASm24DkBjKNgAjQuRlTshyL3OC6Uz+W6VrOe5UxZefhamz5ytpZzeH4dnhT0WvOiW8 624u/6kfu/PlZ8vUL4pTwkcnufws4XQDG1d2+VlG1BnYuEqXn2Uq7cDGlV1+lhHqgY1TXH6W4eKB jdNdfpYYagxsnPjyswzWDWyc/PKzhJYPbFzVy88SoW5g4+odp+fkl5/ZMVwuP+vXwZ0uP+vXEZmZ 1i+fPqYu2zbNfWuqee6w29WbwYjTspowmHWRrPoQTrPqE/FrvVUfYnus+nx79yF4djzzVEN/G7fl QWQOaYm7Gn0Sgi4GT81lTaLx0X38dMBzNfTxK7eERGy35dITzVMRdQXcamuEq1Z0Ir7Wis6N6qn9 tH1zaT/373R0BZrG2aYbp+Pt4VTYb5vvjju8e2GPzpN/LOn0+8in7efp4n8Oo+I0+WbpZIe0RZzq kLaIr3hI29JpD2mLDIWHtC2d9JC2pRMd0rZops4JX+3MhuU6dX7FafJu2drWrOPzafJUGicay9I/ pWvtpSjF8f3He/MY9qtIGjtc0n/nUXzBZuwKBk8Oo3KdGbw1Gp2J+KHIAkGIrYYFIhLNXR0LBFGN OgsEUdgiCwQhNqUFInJQ7yS1QBB8Ft6KTFivuhU5MiyD+lZkYlnzs/sf/99/+5d//n//6/OzOxT+ nk0RyBRh33z984f37z7+9GwU28tSvw6egw/XUfoNyA/Tb5XUdfGMumHGUMncfKd63ji69qK373Nc f5wnF6ePYV6e1h+p1E/BT8c7T/fSMKN+9OFx7vrxfqfTacBO44xVfg/bOLl5EyQfZJzmWg7C1xpn ZDbNOCMzlIwzEkKYfJCxZckHCaNJPsgEwuSDBFZd1xDxta5ruFHJkg+6bezPyQdDPyztc/IBFaGj Nq8Nk5IPplPDXFyw8k0BuglyFSQfZJw8+SDjS5IPEqJG8kEmqpJ8kKlUyQeZoiT5ICGUyQeZQ5x8 kOFVkg8yVWHyQQJVSD7IPKLkgwyVJB8kZO3kg0SqTT7IJKLkgwyVJB8kZEHyQX6/LPkgYWXJBwmq vL230gLVnUqWfNAs0yn5YHVmOSUfUBFMPjgK07Sfcnu5PqAJnR3Ep9x2ZlysYMSYcaqrhCK+4lVC kU15lVBkKLxKaLHCEWPGlo0YE0YzYswE4vt5I1i38bJUOg/vTiVMVx19exoxNs5M2/OJeLHINuiC xWvDpBHjfBoxWtNbN8k3y/3gJemqCadIV034onRVQlRJV01EddJVE5UuXTVRFKWrEkKbrpo45Omq CV4nXTVRlaarEqhGumrikaWrJqgoXZWQ1dNViVSdrppIZOmqCSpKVyVkSbpqer8wXZWwwnRVgqqU yddaa7xTiUaM3Tjbc7pq0/pzumosgiPGP3zb81WtMfdUkP6cCjL7KYiFqQnDmq62Kd+sibhZs1mz jodUErUpYB2XtpopIJKNOSvk6f0vX5d9hMZVEZ1l16wj3yvyqVZ2K2cdZflUEahbs18r3b9zpxLk U41Dt1zyqaZ5Og3vqOg1Z6J1/T2lypprSlW7Kg4KGrd5LDfXZJzcXJPwlG9eckh9RCxVDqmPRFul Q+rzVSWqQ+ojRV92SH1ELNpD6reZNn7Fh9RH+FBjgJepbmO1FPOPZ5Bepq6Td03P7uRpPeGfiumb gJ/fDdKT0TJ8ug2Jlj9/dl+/HoZEuYCagAHPUX4oW4KLD2VLaKXpvNIplHeq4kPZTByNLs3aNU+n slGp29qxmY7Zq/fS+bsnzHfDfsJ87HfP61y+CY1iA8bFMe4o2IAhnFVswBB+X5l6/9MB+f6nhOGW TyOG+jbqUj/8+OXl19S3PvqJazkF6sItYRITrWrFroXefOvtb6/4daIEoh4qz0u2z3mW8LK9i+Op lw+f/Lvtz/HpZarffg81AXsdL+VPWM6fwDHSn8iMchIDnSzJzE3MAn7SV1mgTcethUe00AKdsDUt 0IlQtUIe8bVWyG9UEgt015vpZIFuQjvO4XmgFos2dJf4YbrTPk6IaK/H8zRO4YHuXVK78sx3l67i kme+u3QCdK3M98jmFQvkmSEULJAnBB2g+n57cf4X6nAcHelx68OeC2mI4tgTZty8tkeOMzytXgAg ecs+fX75+s0/gZ/L0hIU+MLe3pYkHuD7y9S+wJ+sS5p3lfT9TnXSd3Ndj6AvdJ5H2a29zqOG6zwK ZVzyzbOfL82zD4rTs4Z1Ea5GLMrViKXqasRSczViqbwasShWIxbpaoTmELqEr7cacT2ETroa0bfB mPAkcqloQ3k+fCtizkad5EdOxK/onawVRZyqFXlXsxV5V7EVeVe3FflXnZHEtyIvPCMpAnWtyFc7 I+lGVacV9dbOp1ZERa9qRfdbDkx31qI4x/TyoeLajotkqJhwiqFiwlcbKiY21VAxMRQNFQmhHSoS h2iomICqoSIxSIaKhNM0T8JXap471V9+qNg/RK47i1wbvPgSEuqBGqHINUqRa6qKXFNT5JrKItco RK6RipzmlouErydy11suxCLXdWtzEjkqKhO58XypGrWiTpMTHnpZK6LZqKYVhb5mKwp9xVYUyaq2 osgnbkWhF7aioDkLJ+GrtaJwPQtHPlQ0rT0PFWMRbEU/hm/f/nxrReNjqHiecM3etmLbbZTbLp3p UT5U7NJRHvKhYsT39YaKkW3WDRUjg5Oep0loSvGOD+xght1f0XCPHXRFUJdA/t3X9eVL+P+b+5Ie TZIby7t+xdyFFszcVu/DAI0eHQaDuQh9L/gqZZcqMzszVd3Srx+S5u6fr4owkqNooSAgLP29iG+h cSe7cQNuJ1TfdiMmhO4Kevr1BF4O8Bff9OkSFMQaHpu/04M/zQW7PwHwdDNFAsE4RWL8gsmC1699 /QzA/PAZjW25EF6w9ScADQ/vkcwwdUpjVjaqw2VwvQvwxexvAuPsNKUx7+dD4GHfTm48jFlZD58a N/daNb20anN1HQdJUdU4s7IMiJNkGRCvl2VANlmWARnqsgyAEGcZgIOXZUCgLMsADKwsA+CERq9S B+dG9fGu404843WpW+K7joOLrPgo4QRGL+HVjF5kUzN6iUzR6CU+ptGLWJbRi0CRngO8lp5bqJSy DKNL5ywDHL3LddxJ0TWVHjrDrng0oZRs1VY8Eo4vRQWvJUXEpiVFhUxPigofT4oIy5EiAkoq2hCv VNG2Uim5jmbszq4jHL3LdXytYvCXXF0zd+xKNpyEHzmuI+IkriPh1VxHYhO5jsTAdh0RXe06Eojt OhKa5zoSlOs6IpjlOhKw1nVEkHBnhFobwUJV5zraZMKYxjYefEc6jS38y36v6Xaan2pb91dCu83p vNk5bBqB9+inwLsSpiC7EgCveCUAm/BKAAbBlTAFxpUAIMGVAGjulQBQ/pUwBeaVMAXGlTBJKlYJ r7ep6Vqx+uaV4HJqeri7/eFKoNNgpi7v40mv08dJYK8rwZmXlXBJdjrTSgJK/cC6EgAnuhICrVlW CygBm+xKCLQulnslALr+SkAQ/0pANPNKQCj7SgAw70pAYPWVEERjXgivFr66GfPy5pXQtNEYOzaH G6FpwzzjnuVxP2J1ORzf4TW4XdfuOcA8udQLFriFPjHvgyS8D5LqfZDE90ES3QeJcx8k0X2Q+PdB ktwHiXsfJM59kIT3QdK7DxLjPnBzxHkhw/FCwNM+Jh/m/Y2wnmJo4M0r4bV4OrXX6f5GMnU5Nrwr ITayKwHwilcCsAmvBGAQXAmxYVwJABJcCYDmXgkA5V8JsWFeCQCsvxJiI7sSYqN2JRSqyiuhwcmj nbPHKwFPXRdjG/dXwnb6OKZ0dyW4V4T+OpPAtez5wRg+9YwrgXCCK6Hgta6Ewia5EgoD90ogdO2V UEDcK6GgWVdCgTKvBAJzroQCrLwSCCRMNGjta1upKq8E07RuSsYdrwQ87VMc7H6k+Ov0aRnt/krY 1XteG+wcP/WNL5RTqUY44ZWgV6lW2KRXAr9SjdCcK4FfqVbQ7CuBXalGYO6VUF2pRiDhlaBUqbZR 1V0JfTRzbrt8uBHwMPama/aBhPXQvSeQELb74GZgrDOChltvOCYC4WS5BqNnIhQ2Ya7B8E0EQjNy DYZvIhQ0N9dg2CYCgZm5BlNtIhBIlmswamXsC1XdfdAak4ZmPgYW8bDHnXj7vQbb4ePK43VFIK5p b9Z9x645JR/95KNj9w7Duz32fX1VT8Hxq3oKXquqh9iGRnAhFAZXt9tsxDnpv/n66eu0+4ZvP6I0 +qtEEY4jUQVYKVEEki1EG5WaFDeqOoladwXmm12Bw2ivuwKH8R0TZ1x8WdzhphacXSaHlYCB1UaM OEkbMeL12oiRTdZGjAx1bcSAELcRAwevjRiBsjZiYGC1EQNOWMWqlV9fqT68FnwnnvGaSTNBkEmD /1gNkIiTNEAiXq8BEv7Ta4BEMs0GSOTjNkDCf7wGSPhPaDlq9TytVEptxL4514Lj0XtqwfdSdG3G nzO/AXLy1vGkCHAiKbJOU4qsU5QiIFOVIuBjS5F1TCmykvW5hFeTIntdn8uvBe+7sxTh0buk6NVR Ec+FnyPWdQikyDCHExrhcEKjOpzQiIcTmurhhEZhOKHhDic04uGEhjmc0AiHExq94YTmv8dwwr14 XmZlOMxBC5RcZiq5LFRyWVXJZU0ll5WVXBYoucxVcpJlWITXU3LXZVh8U9GG/mwqwlGtkjvHQ0CK Rn5RAu5A4bQNEk7QNkh4tbZBZFNrGyQyxbZB4mO2DSKW1TaIQEnqDvFKqbuVSkmKxvY8cQaPHqXo Nf8+bWn8y55HZ/3In39vpzREOzO2jQCusXwhInyo2jaCiFlj2wgQOaezbQSpomzbCFLkqm0jiJiF 20aAwyf+thGEtzrbRpCqr1snRyAcVSZcJ1d4es46uQKdGOvkCBl73XVyRIordCTr5IgEN9+ctjd8 /gLv5Le/Fqr7f0P5cA9flNwJNjoQQ9u+e6NDeX5eVNb0X9OwfiV3ByTND3/t6j3toa+f8a+8W5iS RDuyCK+1uyXd7MgK19UL5fXsa9CS60xoxqbfay06dW2H2yd3NWjr6fRUlvqyAFF9tev6ljafbMAE VmHDrjrx/RwTI9BRcPxAR8FrBToKmyTQURhqAh2EEAY6iIMT6ChASaCDGBiBDsJJstaIV8par1Qf Heh4CWgTYmsv9mWAS4Ubiez7oRkYTlrB8Z20gtdy0ohNy0krZHpOWuHjOWmE5ThpBBTIEeF15Gij 0pqQBN7neUISeIFvhzpOcnQd5eAju5rK2bbhZJcLji9HBa8lR8SmJUeFTE+OCh9PjgjLkSMCirYU A15rS/FCpZQXc86d82Jw9D45yuv8g8Ze4x3B8uMdzo5dZhiMBcc3GAtey2AsbBKDsTDUGIyEEBqM xMExGAtQYjASA8NgJJxsjXinFNPfqNQMxtibs8GIy6/fL6Ck6E5lVNn6PLA9OmcHVhlVwUkU3aBY RkVseopuUC2jKnxcRTfwyqgIKJKjQauMaqNSiurn5mww4lGVoiM5OhVS9d3YW3YJiJ+Miww5KjhR +T0OXdUrv3dRS44KmZ4cFT6eHBGWI0cElJXda83R26iUDMaYznKER++Uo2VAjrM5nByvmHJM7Cyz n2yy2THkCHFBIEeIb9XkCNkmLTkCsrZ5sh27u84URMC3/ef/+Mv0l+mnn6dfp8+rrOyPUFryNU9D +KFdJK08+tP8dSds2xlau3dSgwTTgeCC/i1+Me+h2FWz/z1/nD5P3z4N1z9g+QfMGT18J8auCP3+ j3j9jNmda7qOgCKhB7yW0C9Ux/xCNBepLy/oIvUpX8KWcbiGLW2F1JP2PFVo9dibxK7QSqnrOFZo wfG1Z8FraU9i09KehUxPexY+nvYkLEd7ElC2JL1TskI3Kq2w5aWYH4+qtCfJUThbodM8scMtvp8b zhrmghNYoYRXs0KRTc0KJTJFK5T4mFYoYllWKAJlabRGq/lzpVLy5uY8nb05OHqUo30vdd56qc15 SqMfsfyRu1zP91OYOL3UhJOIEeL1xAjYZL3UxFDVS00IRi814li91ASs7aVGkEicAK8lTgvVh/VS v6oe8zLPLBmbrnt7UmZPQcby0raznOAI4JwkOGJo04lWcATYBtlsgtJhUzObwLR4E5FyQfK4Uzb0 M0pUvvNhAImjTVAU8EH/kgz6EX2mO9fPiDpTCK8W77jpTEkX0aBXcxjuFa2Zp5C77jDcC06H1qbx MAJ0PZ26t0vrffsqrb/ZWO4kpfUzJ0tWcPwsWcFrZckKmyRLVhhqsmSEEGbJiIOTJStASZaMGBhZ MsLJavZnpSzZRvXRZVV78UzXVT/TwDYH8SWOvM4XwIk6XwCv2Pkyj4qdL0Cm2vkCfOzOl3lkdr7M o1CKRj0pGhW9KrApz14VHL2nf2wvRZeSqgZHSbKlKDH7x5Kwfyyp9o8lzf6xpNw/lgT9Y4nbP5aE /WNJr38sqW4sD+ZSUgVHj1I0vgJ8y7yOhBNuzpGJcfJsIXJ2bGyq96QKju9JFXyq6B8jRLby/rFC FDT6xwpVlvSPFYquon+MEK0V9Y8VjpbbP1bgvUb/WKEay6cxfJu6H9PrQyg/0xfitrKrsehsHDqt TgzHc5Tyu9IWJJrKp/ivf/j9v/zb748f4uuMPpJ7ApyBCm/dl6/TZ/xtrzdzO0Gj39wW5jUWI/R0 p+LT3f6SLQf4HuSHP32Edw+DWnCN/fSf37qvX6dvB5r7f0PBfPg6ysrlGqVM5UZ1uHx9e22FKi9o d/v6oYNX0YEft7t/8dCkCKp7dwOvh/3wdnzY2zU+HJpzmgWrw53gFo7w13KK5hAnKZpDvF7RHLCJ 4sOFoSY+XBD18WHCceLDBVgZHyaQSKoAryVVC9XHx4e9dVuVuL/ptkjs4jm4OnybWVXigBNViQN+ jX/9+ecd8s8UejG3ChAwoH3QdPnlj9/g6442zEsfX89RC83XlmZiQqX+E2idV+xn/Qnfq4eXjP22 v4LK/mn+9PnT9z+tRsh2gEJ0De0SFKOM137lU6+yM7eqy9OsqXd3PKdb3ekpAP7rqYv7130Xt7+t ZwUkuko3fcnNdGsxwfPD6gr96H4ctHQ5IPF9+IjHJXQ+77G7A7pU76EyfeyVwucb1bFyKDTXwFx5 Rfvdu7MdfBfHGPYhdDpNYWhAAHa7d9fT8NSbfNDJ20B8WrJ3ukPAsGbH0KfkZoZOLjhBeIHwauEF ZBPp5MJQo5MLol4nE461YYKAtRsmECTr+YdLWKvnv1D9d9DJy7gnF03M584tqo1jylNM/TwyJtUQ TjCppuBrJtUUhMKkGiLSmVRTqESTagpFzaSagpBNqiEO/qSaAleZVFOoKifVEEhhUk3hYU2qKVDO pBpCak+qIVLppBoiUZ1UUxglk2qIoWJSTXmeN6mGsJxJNQQUaC3C62itjap6Uo2b4Etphrk7TKqB 026ec86HgortdH7aqfoNU/Gb+lon1QR77iPx2TQju+Qopa5JznAqygHXsNVXwccK9UUIb+TqqxB5 DfVVqJJEfRWKtkJ9ESIYkfoqHJmrvgq801BfhWqoU18EikmsvgrPwFFfBToz1Bch06CrvogUjViJ +iokQVF9FcZeoL6Ioeverb7o+XclLYJ5+G3MpAVhn5IWzymL+4QFkck6QhqlgVAb1VEh3qYrDqni FEAxTaeyJd8O57IlOGrfowSb12rxNl9iIq1hFy6ZyfuOoQQLjq8EC75GCRJCQwkWIhUlWKhESrBQ 1ChBQgiVYOFgK8ECV1GChapSCRJIQQkWHpYSLFCOEiSkthIkUqkSLCSaSrAwSpQgMVQoQXqerQQJ zVSChNVSgkQmG1zvlfKLG9WHK8Gt+cSEdFGCtmOPcrO+6RxDCRYcXwkWfI0SJISGEixEKkqwUImU YKGoUYKEECrBwsFWggWuogQLVaUSJJCCEiw8LCVYoBwlSEhtJUikUiVYSDSVYGGUKEFiqFCC9Dxb CRKaqQQJq6UEiUyS0EO8UkJvpfpwJbjsnXDuMhggxZhmdjgUp1UHhhIsOL4SLPgaJUgIDSVYiFSU YKESKcFCUaMECSFUgoWDrQQLXEUJFqpKJUggBSVYeFhKsEA5SpCQ2kqQSKVKsJBoKsHCKFGCxFCh BOl5thIkNFMJElZLCRKZKD8IeK384EL14Upw60DL2V6WLw0zezi+s9FZhhIsOL4SLPgaJUgIDSVY iFSUYKESKcFCUaMECSFUgoWDrQQLXEUJFqpKJUggBSVYeFhKsEA5SpCQ2kqQSKVKsJBoKsHCKFGC xFChBOl5thIkNLuRCbBaSpDIZO0WTq2JaaH6aCXo1gGrOTl3HrA6GMffbJEnVmFMwQmUIOGrlCAi VJQgEekoQaKSKUGiqFKCiJAqQeLgK0GC6yhBoqpVggjSUILEw1OCBGUpQUSqK0EkFStBIlFVgsQo UoLIUKME8Xm+EkQ0VwkiVk0JIplECSJeSQmuVB+gBL9//1OZokALnZaqGHueRteOaZZUxXTY3smo iulw/omgKgbwXqlTqLC1gk6hwtCV6SNH+P6Evq63JRLdhPPlSE6+foHv1M/z+H0nO9vZb7Fv+9p3 VBjaHcMJjH/1w589TqUW+4VafyI5vwfJMu5ag4g3qoN0NeZaiI0vZz/Xzpo4tPPc58PCUDx11s35 sDB0O30StH1Tns9LFbb18VyFHQccNMY1Nts+9ZxGecIJGuUJr9Yoj2yyRnliqGqUJwSjUR5xrEZ5 AtY2yiNIorQQr6S0Vqr/Bk15wWy1LM1pMLEBfKYhotxBqr6td94Kju+8FXysapQvmEGhUZ6YcHJY TaN8Ab3Vam7d/cxVX9FqXp7ntZoTltdqTlDhpFaluVsbFaPVHMPTM3y1jq3mdIpXpu/2zUXraXga /7IzH8Nrjt21qNobtgwa+P+26+vNR8KNfPMR8b3VMh+JLUrMR2JIXPOR0KPEfESGIVabjwgb+8V8 PNmPCyzdGJAIkxiQiFcyIFeqo7T5q4P29dGCjHcWZLR3FmQMD7L2t0XhtWsPOs5Fvoia9Xx1h1Mx HXoojMGrrh/4pZuIH4ySqBW2IBC1wvAYO32Yi+zG1TsD3u9/+Q7e+rjXS9shhdzuZ7xStRdNgDxR HM9Qcd+UYCKBcL6qVoXXSnUcTnadJf7t049966u38DU6BDRsM6fxGNCgo/zkZ/32tyQlwdi12dU5 my9bdqcxsZd4+sm6LkXO0jTAZYFZiPih0iwEDM56UjALkWmsNQsB1DrO/KQCjcz5SQU9yOYnEQku Uqien1SQ91HVe6MWnn9XVNXeSD6hmVFVwmrOSCRC2TY3p9RtsVHVz0jMzdQ2bT5eRslHk/LhMsKj bP6+ixqwtGa5i7xpT6vc/ARqnB1dTamD11g/obbg+BNqC15r10dhk+z6KAw1uz4Iwdr1Qcj6XR8E k/XiWiX9vFFV7vrIjfM2HRZ9gG6ZmhiaY84Bj8YHqfil+3la5aLdFpu2Pp4HUnQx8HdKTaabeDtw ACfagVNWK2rtwAE24Q6csnKxZgdON3F34JQcRZ1cEEwWWdFKFmxUlXLRp8lO89geJIMOY/B92snG ejg8bQD9Dnp++vxP/nft7wxJid0MWXD42uuiKGvZQ8fM4PrMcPgKju/wEV7N4StsEoevMNQ4fIQQ OnyFQ+DwEYEkSIJ4pSDJSlXp8OUmmNh0RzPL9LNP7VGh4NFjFPIqMo1ZRSaZ8+4ojJGwfT/rm5YT jiw4fjiS8GrhyMImCUcWBm44sqDfsTljum2RbVvW5gwEyjrgWqWY4kbF2JyBAnPcnEHS0ZqLwHRv hRJBTraqj2gvctJMhq9aJp97jmohnEC1IF5PtRCbSLUQQ5VqQYRUtRCHRLUggayAIytlmjcqeSwR 3b1TLBGPnmKJ3/8yflnkxG1ykprruISY+HOUfcNKbxWcSJ8oprcKm1CfCNJbBS1JbxFDfXqLYPXp LYLJVJFWemujqk5v5caNzh9yW6B3fBPgLz6oIjx6Wii4l7FtLldy14WCI39NO7zE2fFkbHYyGZud powBm1DGZieRMUALZazk1aplbHYsGZOlwxCvJmM36TCRjJmrjJlHGVutPedeXtE1kJAtu/AQ3L/Z MqJtBcePthW8VrStsEmibYWhJtpGCFa0jZD10TaCyYIGs1L310alFIXOjT9HoXMzvyEUvm1faudi 2lmf+HtsvRsjTvGuVjuIiwK1g/iuKk1MGHT2xGniwtTXpYkJRInW6jRxgV5npQOmg6/fnz79edwn je/+HYnv1A4Sp3e21mTz8Kc9VDbeTxrC53mVjYTlVTYSVKQoAa+lKBcqRmVjuRXC5VZIfjzfCilM b2RsbWjD2hHqgrmMSjcDezaQh0uFl5lCnCQzhXi9zBSyyTJTyFCXmQIEMzMFSE5mCmCyUgavlpla qD42Y2ujeXlp5xUCYEM2Pb/6cIqsSoaC48tFwWvJRWGTyEVhqJELQrDkgpD1ckEw0XKoqFXJsFFV yoWdTaTpD4eFa8YPTbL2sG6tnD0tW1sUhgtxKzds87mUAYfJtewQIYh+lxmb3AuO39VV8FpdXcSm tcm9kOltci98vE3uhOXkowgo0y6dUvp2o2Lko1C/nPNRoExcvugX8Enuheg/XgqmMetkYh/dqby9 6UGSOraCcbbNjrVGGHGSNcKI11sjDGzCNcLIULdGGBGcNcKA460RRmD1GmEAiebaAF5rrs1CVdcd mZvgenPojcyNNX2G/w5ZKzx6MtZWF6ZxJr/CfdekVWP5Lf2DmyMz3BeF4b6oGu6L4nBfrA73RXa4 L/LCfRLnnvB64b6rc/+2qdaEMcbWhXww1ug0NQ3Omtu1Ua2nzRs12bYNYd3l2frUnGuyfczs2gdn 8+QYBlvBSRQN4vUUDbBpGWyFTM9gK3w8g42wHIONgCIFA3gtBbNQfYzBtno9zRoN8DGc+xpsn7sg kKE4e5YMIU4iQ4jXkyFg05MhJNOUIeTjyhBgeTIEQNnwwVmpjX6j+lAZAktx83i8bcK1CC+yIwcG 02sMISo4vhAVvJYQEZuWEBUyPSEqfDwhIixHiAgo6o63WqUNG9WHClFo8mrNmcY2l0pWlxO7esjZ 0YyssAHiJJoI8XqaCNiEYQNkqAsbIIITNgAcL2yAwOqwAYBEGgnwWhppoaoNG6AkncIGmOZMlxxP Gp72RK+S5Jq8BrJxwOCpb36a88AOG4DCNS0jbFBw/LBBwWuFDQqbJGxQGGrCBoRghQ0IWR82IJjM TDNKBaob1cdmPl3TLhomNZeVec52oeMXCk2xSbzMJ+BEmU/AK2Y+gU2Y+QSGysxnk7iZT7gMOZlP 0dhYwqtlPm/Gxr4tF23XjXPrj72qeDjOvZv2bXfr4fzGCL7gvFlcGJdIfVw2abErS30/tYFTLUM4 QbUM4dWqZYhNVC1DDFXVMojgVcsgklEtgzBJPhPxSvnMlaq2j3vIIXQYaN73ccOhbx2Iwb6Pez18 o5KsMTGb1b2PZ8/E5TG7xF6w0/Ytp5Ks4AT2FOHV7CliE9lTxFBlTyGCZ08hkmFPIUw2ubVVqiTb qD7UngK5aLcxyG2+jowMvcBjz+3I8jMQJ/EzEK/nZyCbzM9Ahjo/AxBMPyNjU3a9nwEwWUqlVXO+ F6raSrIhWOu7fBQNOm0snO5T99tp/9SavWZWUljFw8FXzZ5X0uQ4sMUjgpXOcTcKji8eBa8lHoVN Ih6FoUY8CMESD0LWiwfBZPsKByV3Y6OqdTd8M8Vhbo7uBhy66AYz7N2N5fC5V3Qxqax327Th1F5a RV0c+dvrp5h7nisOOJErDnhFVzz3UlccGCpd8dxzXfFlZEOtKy4aUUB4NVf8ZkTBO1RHSmYKkx2O qgNPm9zHbj/S43X6VIr89ec//hN2xXwaVjHxdo1YpevUQfBG+KWULbxcnucxS2r1C17R85hltfqF odLzmHm1+oRkeR6iXCHh1TyPm1zh22LSBTfMR7djHPrUHd0OPHpyxq+iEbaFfzleNUhrBVXG2Qxh 5jgfZohW4nwAPlRtvUXErLH1FoiS09l6i1RRtvUWKXLV1ltEzMKtt8CRE3/rLcLbw8K/At4d0Ft2 W1sHWLiEfvzyFd7n8qt+Xb4BxzNkuF2UCwy4l648XZpa7Z5gOcIXHx/ewKphy7cbYoEEdYrihlhk fG8H7/2GWGCo6OAtz/M6eAnL6+AlqMyLNoOaF12oGB28cM9nd77nm3E43/Pd/Dg8c/We29X08fCr z8m6PIIbyx0A6OzgLKswEXGSchDE65WDAJteYSKSaRYmIh+3MBGwvMLEQbgVedDbijzcbEX+x9dU rSshfWPOXrazfm4FMtTmkSVDiJN1Yo2KMgRsejKEZJoyhHxcGQIsT4YAKOzAUovmLlQfK0Pg86x1 iSkke83+JXaFvO+nxAnjFpwoM570wriFTZgZT3VhXEJwM+OJEcYlmCwznpTCuBtVbRNWHsOQmzYe m7DwNPqc+/3kpdfpG8OXgk9xbSABib0u6RlGSRNW0/GSgIATJQEBr5gEbDppEhAYKpOATcdNAmJM kZEEbCQbYwiv5r40140x74nkutyA2hhPkVw4bZou9eMhkrucDo+DZ1fxyHExwWxrc7hGcD27dgTu gT439Vu5Cef4W7kL3leEqQpikoepiAhsWYUwVaEKkjBVoUgVYaqCmERhKuLAQlVemKrA8yFMBe/U 9+nbr9dw1foPaGU+fBqhK5/G3wD2+gjwJ/z63OwgQxAGOfGx8dO36fOPQ6Tpcow08VYh41LGjQcf PjDgAV5WT3/CWKDn3/1aSTY8vOLUHb9/FIYazzx3/wqs8WaICJLmSRZ6IxLc0nY7I++XL+On+a9w P92G4E7PoLzcmx3wG3pBKI4YHjaq3YXi6Hn+RjVEczeqIVZ1oxoSysxHrT0fGxVnoxp1qZiLX9WF i1/Vv9WlEraxnS7HZC4Vx4E/3MLZ0c0swxFxEsMR8XqGI7LJDEdkqDMcAcE0HAHJMRxHLMCVtG65 Wa11q1BpVVWew95wNL1RGBNCdq+RL2d3yoxmmtnuFG4smRghu4Ljh+wKXitkR2xaIbtCpheyK3y8 kB1huUtxJIXJhNdbinMtTP6Hhey+/vXHn758dr9rSZZya1y71icHHJx+LsAM7PB3TP3sfGYUYCKu Y8tSwU8Vvhch0N6X+l6FaNDwvQrVLPG9iCLaCt+rILLI9yKOZLi+V4E3Gr5XofJ1vlcBzWLfi3iy 5/heBZoYvhchW6/rexFpl2W+VyEZGTumCflQnHC3Y7o8zytOICyvOIGgohpmtQK0jYo1Xjz4MJ3H i7d+mOajjmn9aJo3DLUY01LBnHDz7Snu3bbzwK5gxgB/AyLASQs1GGYTpIUajK7VBPYAMagE9lIT jFJgD6icMLAHFKEusAeIQRrYS030gsAewKNSYA+ocnVgLzVrCaEssAc8mRnYA2jPC+ylJmf1wF5q pDV1RKJaU1cYJTV1xFBRU1ee56ktwvLUFkGFOdtGL2fbKG/FiO10cY26p3TtawpGjHnt2gw5X6Zg 9LZhx91AOWdO3K3gJG1piNdrS0M2WVsaMtS1pQGC2ZYGSE5bGsBkJl1WirttVLX1DH3I/di1x1Z/ Oh3MZOPeqNtOsdns79t1Ka3N/tHaa1NzjIKFuoHV7F9wgpnLhFebuUxsopnLxFA1cxkRvJnLiGTM XEaYaHlu0IqpbVSV4hFDM4BF4w7SgYdpAoWyVx3rYXhLNpKNq2wE584+D2j+hj3n0uWlJK02ZUM4 QT1pXkrPdFI2yKZWT0pkivWkxMesJ0Usq540y+rmCK+U38l3dXP/uHrSRYpy266JT+9jPJcEjeM4 szvX/GRC2zFKghDXC0qCCO+1CkqJrRVomMLQMdfkEnowhr8mtzC0tWtyCTZORcpOW3ILKNw4SAgS 7aABvNYOmoXqIGGNuZYWnHbkznOXhzTbvBcxPGydDc1+aNN2GN/SVT7ENZEa28tMZmf4a87M4HpO 63TBiXZn9Hqt04VNuDujr2udJgR3d0bPaJ0mmGx3Rq81Znmlqi0v6IcO6+uPBQZwGNthOG6VWQ7t m7KR/aKBYmPMeaxA22ClEr+3buYNiEWcpPQG8XqlN8gmK71BhrrSG0AwS28AySm9AZisXW5WGxC7 UH1E6c0rNJZtWnRGsjm4y4DYlj+xycGXnjMEs+AkcoF4PblANplcIEOdXACCKRfJMYZgEkwkF4DX kouFqtb392NMdpyOvj8chjxNcW9PbYdv7SLLjfdrvtPls2xMc57Zvr/1zeQ5E5sIJ5jYRHi1iU3E JprYRAxVE5sQwZvYhEjGxCaEyWrNJqXdLxtVpWwMTedNPk0zg0M39c1wmEizHuY3Jv2F7LYeuGhN vNYC8H0NDFo0HL1BOIHeyEuSUC0u1sj0BjFU6Q1E8PQGIhl6I8vSjYTXC3Vd043/oAGxi1QsM2HL 8LJsT2OTQQn1bWJHi22ceNYU4kRrKCZNawrZhGsoplprChDcNRQTy5oCmGwNxaRmTS1UH7uGIrvk VrkAA++61XUWeBkDaw1FwYm8b8U1FIVN6H1XrqEgBNf75qyhIJjM+9ZaQ7FR1Sbgu3buxjQdO6bp dBhdmg9bXbfTNytUQHFsCZJo7lyNnh2eMoObOOZUwYlCt5OeOVXYhKHbqc6cIgQ3dDsxzCmCyUK3 k5I5tVFVCkjIsZ/G9hi6xcOxjdMcd+KxHOKSlb9vVDU2+XVPi3Gn/jCfQ7ZsN9xPIHicaTSEE0yj IbzaNBpiE02jIYaqaTSI4E2jQSRjGg3CRDk/wGvl/BaqDzWqfAP/W7uJ02VcoO2HSdBNnCbWjhbC iUK3k+KOFmIThm6nyh0tiOCGbifOjhaEyUK3k9aOlpWq1qhytpnaNB7jU3SaBjeHfWJjO41PGb8/ /fjxdVzcjrQGcI217lzYOJqJPzDZDE2MOPK32qpCXBZYVYhfv9N//nmH/PPPhLmWSxAGrpXfYFfK L3/89tN/UXvKq9Xieg5Mrr+zCpAJ5eHXzz/hw4tELD/he/XwkvXmdOSb9cj0G947pyObO8sLGCrm dNDz7DkdhGbO6SCs5pwOIhQZnYDXMjoXKs6cjhjgG3ua05FTzP1Rt8JR+7QZ8PbqiOZasTaBFPAj FnmymVf3CThR3SfgFes+bVas+wQy1bpP4GPXfdrMrPu0koE3hFcLhtvrwJv31X2iHB3rPr1rbD8d Om/oaH5qGH2Uo5ORGmafe3ZvQUz91DLkqOD4clTwWnJEbFpyVMj05Kjw8eSIsBw5IqCsT2dS8vQ2 Kh058rnxJznCo0c5WiIgrQ1rYadLyeRr+fQoKV5rDW8wPOBEg+EBrzgYvjWKg+GBTHUwPPCxB8O3 hjkYvjWyWHtr1GLthepjmhCG3/52UUVU31amWrf5PNXa+pHvDfp+siPDGyw4vjdY8HXeIGH23uD3 B2/w+8EbnG97z4Gp0hskEHqDv/5x+oH+3afvywveHWCI5LZNHqBxdfOuTt6r393ctkYDuqY1Pd0O 4ba0pLV67klB3nuYd3NP6HnJJEg78idB2lF5EqSVDNgnvFpTur0O2H+nhzkMsTl5mGnIbjhGb+no 6TLqv3z58f0HvGeLYk/rtgrjmnRO+7VAxJ6oAi6AYYVwCSeqojKKIVxiE1ZRmcoQLiK4VVSGE8JF mMxxNFoh3JWqMoTrTAiNSX1/COHiKVx6Yz/sE3+v08ehdsfFd6E16z4K59vmPHAfpGjiN+HaIQ28 8hHAicpH0qBZPgJswvIRYKgsH0kDt3wkDazykSTbFwZ4NZM2XfeFcXZC2sZbe5zFhUfNU4jyIhpg zC7xFWfiuQ7X+n7mu4bg+XeJtS4VcKJ1qYBPNTshEZGtwk5IIgoqOyGJKot2QhJFV7MTEhGtle2E JI6WvROS4L3GTK5CtQytB2O3+zG9PoTyM30hblx4RHbn2VYnhuM5CvqNG01EU/kU//UPv/+Xf/v9 8UN8ndFHck/wLjPem9u9u2PHNOMJq2nGE6FsJ++otfBkpao34+EK7tLpCnbpYKAsR09rebe6pBTb bTSIcZeo3MTfkhVTHzk1ewUnmioV9Wr2CptwqlSsq9kjBHeqVGTU7BFMFq2OSjV7G5VWXdLgLmG2 R6k4DqIG4dhWHUTbtpcdWW5kD6K2UzKZ0ztHOEHvHOHVeueITdQ7RwxVvXOI4PXOIZLRO4cwSe8c 4pV651aqauGI82xOwmENKPX2qDLw6O1OiNbHbdlBONvrIGAzX2kY7PPhzVrrRDM6CK84a60Tzugg hspZax1zRgciWbPWOtGMDsSrzVrrODM6mnkYQje3x5o9OPTOz/4wiHA5fHPWWhtcWmcROHv2ZfMY M7+n1IJLwgrzIE4S5kG8XpgH2WRhHmSoC/MAghnmASQnzAMw2TreXi1zuVDV7lUMCTyvpjuuHaXT rhlGt187+jp96hIadtojhvAqaD25HAZ7jdhVaWZoPHqqjIJWv99OzSho9WNv1dqEkC2K2oSQITHn qBX0O3L90zViRFhOrp+AsgpOr5Rf26gYuf4e7v9Trt9m7Aw92ll4NLyV629TWlSJN7lNl/JNP03c HaSgmeaWkesvOH6uv+Drcv2EUcn1F6bKXD+BeLn+AuXm+gtamOsnElauvyDfn+un59m5fkJzg4SI VQ0SIqGs911vwla8mbD1zlz/ZGN3zvV3LvTHiAgd9U+X0bcvn0uaH975dttw56/t74Y/fM5MYG33 9fV7Bcev3yt4rfo9YhsakdOHDK7K6UMEvIKvn75iIqJbJG77EYXbT/e4EbTH+AUF4wV8/QzI/PAi x7bo9xds/em3OH3nHiTyFNEbU/IUF6qDPF1VO76YnSwla0CSXMzDTprw0M/zMNqdPG2Hj8nPQ4wR xCpv7V3J2UssZerZSt76Zs4Ma7jg+NYw4dWs4cImsYYLA9caLmieNUxY5m7IWRRzRLzavK6Z24aR u6E5V772eTDHRBUdvbOMJprGbd5jzs2lVmCYJdOJXORNJwKcaDoR4BWnEwGbcDoRMFROJ3KRO53I RdZ0Iifp9yO8moXmrv1+rDKaNE59d3QU8egxW3URDZ+2sPx5tR1YaCmzVYmZAjitgROWB1wSWGiI 7yvKaAjRBnkZTSFqNcpoCtUgKaMpFFNFGQ0huiAqoykcI7eMpsBnjTIaokKbAt/0X34eP317fQb0 I92Gt0kKwJ22357wh2N8JQ9vf79svv2//+d//e8/HD+/7QjfyIe/YrDL/Ui/5+DeLif4GuxNaAzR skRLY9USLYXq5I7Gy3W3vKTjhecvF543w/nC88G+UbTi2rgNDXH+2NfcgDkejMAVDYGZfwzC/GNQ zT8Gcf4xVOcfAzv/GHj5xyDMPwa9/GNg7QjQHabj2ja8OjEuu22zE8yztePE2w+AOIltPE6a+wGQ TWYbj1PtfgBAMG1jQHJs41HUekx4Ldt4vGk9fkfuMXeTn6f2OEyHTqfJtXYvHa/Tp4TKJiAR/qK1 e7Ix/mwhjybzVzg520Zmcj4Kk/NRNTkfxcn5WJ2cj+zkfOQl56MwOR/1kvORlZzvnMtxmPxRQPA0 DHZu9kbVdure2C9jrTNLXZcz4VwMnGJMbeI6kH4CK5w3pLCXtCgVvOKQwl7WolQYKocU9rwWJUKy hhT2khYlwqsNKew5LUoqxcCLVIDSWK0qH01z2RIwTJFdBezsMDAb9wZh496g2rg3iBv3hurGvYHd uDfwGvcGYePeoNe4N3CkwjUBE/rJHvtb8TR3cOUf+lvXU/uGMx5NaNrNprKX5bKhC4KAfB46lngg TiIeiNcTD2STiQcy1IkHIJjikVGW68UDYCLxALyWeCxUteIx5dmnyR73ktNpyGnu9zbVdjo8bSc7 B+et2Qa3OJfOvSTt2A/smnkHDqnliAnhJD2uiFcTE2ITiQkxVIkJInhigkiGmCBM1H4IeK32w4VK IW81zE1zCuPC0ZO7cRWNFF7tJM0lb5WjpLJotJ5REEw4QUEw4oOtylshotfIWyHRrJO3AqrYyPJW SOGr8laI6IV5K+DA7n1u3grhQSdvhVSpfBp/A9jrI8Cf8OtzU4CNIKx4xcfGT9+mzz8OtZuXY6SJ dxFy5EkbDz58YMADvKie/oSuQM+/+1W2Ojy84jad2taxUHU889z9K7DGuxQYTjbrK2phuwcRwwrV 03zmz1/gnfz210J1/28oH+7hi9LHd85sDjez8YhhCO+f2UzP94vG+guO7/v8/ZBL3M5IhTxcTeuy 8hPB4Qhf8wNcVsunpcE2qoMGC/6qw7bXdNRjsT/rMd/Fsx7zjw2RiweUUoqL/nKuaU8BAt8nsBTZ OwzAYeOs2Cw4kWnX6a3YLGxC066rW7FJCK5p1zFWbBJMZtp1SkWuG9WHpiNRLtq1FTKl0/jxmHLy 7GyLbdLMap8nnKB9nvBq7fPEJmqfJ4aq9nlE8NrnEclon0eYpJQV8UqlrCtVdZswwHKK86lPGE7j 7FJvD43C6+lTS8WabWmauLZ3gY19Hghnu7aXBM7CwBqIjDjJQGTE6w1EBja9gchIpjkQGfm4A5EB yxuIDEBZlC2oBaEXqo8ZiLzZXrnZVjibEM+j+UPP1zG+nxwnrFZwgpQl4dVSlsQmSlkSQ1XKEhG8 lCUiGSlLhMmG8zolp2SjqtUxfRfi5Lw76hg8HdLcxH1d+Haanjacv9rtmxxTs+VnzKnBqDH9YPnj eeFibBkNRgXHj64RXq3BqLBJGowKA7fBqKB5DUaEZWqSLCsSA7yaJsnc/RRDc95P0QCb8/7Ubg9H +W1/JYfVX2myv0zobeeRXf7iSOF1nF2E7d5zrpcUwGPIS0lSkC1oGWRI1ioaZMCHqoOk5usX+Nb9 PI/fd5K0naHyae9kAhnaHcMJjMJ/u1+x9WtA7YVaf8JvxN1QAwDJNh62WtGCleogf425tpjjy9mr rKZN3rt5ynsJpFM7hnSYefE6fYobfP70+d+7RWWlZu2J9emcEMJR2QO/TLPtA2dCTMEJVBbi9VQW sYlUFjGwVRahx/fK2m14DhiGWC1rCBv7RdZOwrbA0u0Q3iAZLkN4tamv4TpcJvhwlbaTuIHf1Fgw 545+0whez6moDY+expStw2Wibd2rqu08XCaCeTbxzcK2c5y1ogUnGC5D+MrhMojRGS5DTLXDZRDE HC5DUPZwGUJLh8sgCW+4DCErhsvg8/zhMohmT6AGrO4E6k7Wuox4tTzBTevyO4fLzKM9rypN2Zl2 Og6XgaPujbpzuIuCXQ3v5jLoKhszC1zUsZ0Yo2UKThIJRbxeJBTYRKNlCkPNaJmCqB8tQzjOaJkC rBwtQyBZs1Ortjl8oaobLVPCoYfBMiX2eRgrU46mJyd2p9bbpczQ2sZcVmlMbpTEQ7uJtx8OcKL9 cICv3g/XTVr74YCpfj9cN7H3wwFUsB+umxT2w3UTdz8cIKv2w3WTZD9cN/H3w3WT8n64TnIREV4t BN1dL6L3qnXfhLNan5qcj3NJ8Kh9Y+OrDzGuG19BwedL/j/O7I1XMXUDb7cE4iS7JRCvt1sC2WS7 JZChbrcEIJi7JQDJ2S0BMNFuCcBr7ZZYqGo7A8Y+h8F0xznhdNr50TfjvjNgPXVPunoRj6Zt2q1s LOV0aZzpZ0FfWWp6Vv4fcRKrF/F6Vi+w6eX/kUwz/4983Pw/YHlZGwDKosZNrxY1LlQfk/8/9xA0 Zh2j6FI4d6GZ0UwDe/ZVSl3DadIsOL6uKXgtXVPYJLqmMNToGkKwdA0h63UNwQTyQXgd+dio5O01 tpnDaUoSHEXzlvmVzGvYRXPey2vDkPhlyTbPideYCThRYybgFRszgU3YmAkMlY2Zc+I2Zs6J1Zg5 J1mif05qif5C9SGrva4KoyT7G+xxvtnT0rJjjv2QS3C7UjoKjp/sJ7xasr+wKSX7C5lesp/4RMn+ wlCd7CdYbbKfQAIZJLyODG5Utcl+XDnZn/VSO/izXmofO6K3GGXjsdq5pB5zONfZ5DF4tuPjJ9Nn ToyScIIYJeErY5SI0YlRElNtjBJBzBglQdkxSkJLY5RIwotRErIiRonP82OUiObGKBGrGqNEQtFk H8BrTfZZqDgxyrEf8zlGaYIz4zFGCUf2qUB2NZKjWXuUHFxGl3HiXZ7Y+ZKYelaPUsGJ9t8q9igV NuH+28oeJUJw999yepQIJtt/q9WjtFF94MgrH12ym+vowmUig5NsnmodZ8BuwYnmJDq9AbuFTTgn 0dUN2CUEd06iYwzYJZhsTqJTGrC7UX2EVLzmhzZtei2+teYiF35kh1TMFCInylhwosHTUS/KWNiE g6djXZSRENzB05ERZSSYbPB0VIoyblSVcgEeWNP4Jh6Xe9Jp0w2u3Wd7t9POv1HH5czaQOF9aE6t Rt56gTEFF8A48tQG4ERqA/CKamMcpWpjHGvVxjhy1Qau8mSojVFUbI14NbUxXout39OM17vcz2M8 NePBaZdb46dDM95y+rga+qU/3JalSk1jrzv0bGavvvX9zNIfBSfpVp0V9Udhk3WrzpX6gxDMbtWZ oz8IJioVmrX0x0ZVWxHRhQj/NUcBoVM3dc10qIhYT+enyOCmP2y7tqqCvXetiHCTpCLC8/YXIE6i PxCvpz+QTaY/kKFOfwCCqT8AydEfAJMVOXi11tSFqlZ/uMmOY7ThqD/wtB+9N2GvP9ZT+5TQ3QLn jorjF/k4FzlgK3cvaFCNY4wTZ/fdGJMRyAfifcWgxIKY5IMSiWidsicblFiogmRQYqFIFYMSC2IS DUokDozk8wYlFng+DEos4N0BvWV3BVmIhbvsxy9f4X0uv+rX5RtwPEOGm9mKxABOyYEB4+FXDjxF Fn+7FBxY+pVl+DZ1Pw4M5QTfwIfPoJvKB/+vf/j9v/zb74+f++uMPsV7gndlF7y53bs4shubEKu8 NXsU7mQc9XYyjuzGpsn7y9ZscNrSqQIajvJTBfS52MD5dXe2D2DkXOz+yJ8QGFMXOIZNwYkqoYOe YVPYhJXQoc6wIQS3EjowDBuCySqhg5Jhs1GJq9PMNKdwLtyEo6dlTFfRWG1+b0M4x4zGccrsJgHf T9ky6nAKjl+HQ3i1OpzCplSHU8j06nCIT1SHUxiq63AIVluHQyBZj07WGxOVr2OiWHU4IG7wEZwl 0A5PC+hf7kQwftVK5hKuHefMj0bB/ZIaRh1OwfHrcAq+rg6HMCp1OIWpsg6HQLw6nALl1uEUtLAO h0hYdTgF+f46HHqeXYdDaKalTFhNS5kIZYZA0muJSteWqPdayp27WMpz4/18tJThKLxZFOjC2rjs fXb5kjtqDdsOcHbMgRXbAJwotoH4utgGIlRiG0CkFdtAKmFsAynqYhuIkMY2gEMS20A4N7aBWFls Y8Q2QnFsA1nYsQ0Ei2IbQCCIbQCaHdsArG5sAwhlYyZyUBszUagYN7YzuWvPN3bXTk043th49MbW A6Bya3e3bVp7Ttb0g2Ty7txxKicLTpTL7PQqJwubMJfZ1VVOEoKby+wYlZMEk+UyO62S4pXqQ7ce oFxsNWLJndruTD/Olp3EtN4ZztSDghNsPSC82tYDYhNtPSCGqq0HiOBtPUAkY+sBwiRbDxCvtPVg papNYkbbmuSH4ZjExNPocxv3LanbaXpj1pdPISzxPhfiebYn7pju2FsP/GTHwNsxDTjRjmnAK+6Y BjbhjmlgqNwxPQbujukxsHZMjxJ7ivBqO6bHqz31j1YbIBfZrv5vex7chYsMnCRF5GdeighwohQR 4BVTRMAmTBEBQ2WKyM/cFJGfWSkiP8siQ35WiwwVqkq5sFPs2tmb47AcOp1AYqa0k47tdH5jy7RP KcQ1Vt20+eJt2EEQHsojrzQMcaJhBqNmaRiyCYcZjLWlYYDgDjMYWaVhAJMNMxj1thaMnAyqZp9W 9DavPnhszgWTmMFJ7M25IP42J5bSAJxIaQA+VQRNCZGtPGhaiIJG0LRQZUnQtFB0FUFTQrRWFDQt HC03aFrgvcbm3EI1lk9jH7n82z5u2YzX6Cshu3zaQHtiOJ6jjN8mfoBIEAElAnYElND8nJVV7R0n QpllYtVaZBcqTs6Khu2a8+2bTr3jeGSfSljWCKiN7XL7JhfPEVAzuZBb7u0LZlLgdMkWnCDSQ3i1 SA+xiSI9xFAV6UEEL9KDSEakB2GSSA/ilSI9K9UH2iQ2OLsVqafz1Fe4QmzP7h23vhk4G/kKThL/ RLxe/BPZZPFPZKiLfwKCGf8EJCf+CTBR/BPwWvHPhepDAzwoF0tewLvWnBeuuNz37LyAmXDcIKd3 HHGS3nHE6/WOI5usdxwZ6nrHAcHsHQckp3ccYKLeccBr9Y4vVLUBHtumGNt0zAvQaQi9Gdp9gGc9 HZ+qEdcAT17n8cEX1QR/nYY8sl1ZXB8yccSDcALxILyaeBCbSDyIoUo8EMETD0QyxANhEvFAvJJ4 rFQfmxfIZq2Ptynd5QUCe06ln0zipJMLTpIvQ7xevgzZZPkyZKjLlwGCmS8DJCdfZkQVo4RXm9x2 UzH6DrUxz0Mz9v0pL4CngxvGeS8dr9M3fHB4i7d0ckzp0lllfRRMqoosq6rgJHmBqGhVFTZZXiBW WlWEYOYFIseqIpis9VDLqtqotNTGOFzXQD1VU6Pa+M8/dT8+fV/yAyEsk1aN8+GkPHwaBv6QY9ei UcdYMUE4wYoJwj9+G++KqhHhk0JRNRH1KkXVRDWJiqqRIpiaompCJFlRNXHM7KJqhEerkR8oVK58 Gn8D2OsjwJ/w63O7uNZgkzKBxk/fps8/Dl0rl2OkiXfLN4EnuY0HHz4w4AHeVQ9/ArZG/e3UMfO3 fceMGx5ecXanvAa26Ixnnrt/BdZ4t98SSNtU0QXUPYgY1ruXR+ev5WFsXur+/NMvX8ZP81/hniqU f/8ZlJf7tblmxHr4I/qOES+su6UryNC7234j2z+82/3amAhfCfjnQ/5lPUI98PCW4I6Xc5KlwO4T MOXf6Dt3TyjbYWz0dhib61gtixbcZdchvqC9Fei6qZlDbuZ9DoZOh7Hv2r2y206Ht1Zd5CaHda5W as81twPoKXbxlJ0i/AWcTAziJJkYxOtlYiK9j5JMDDLUZWIAwczEAJKTiQGYKBMDeK1MzEJVW3Mb +mkIeczHmls8baa5GQ+D59bT6c1l3zhnf4k9N6FJ16LbyJYP389gU3C2giJOshUU8bVbQQGjtBUU maq3ggKIuxUUofytoIgWbwWdS3U1YysoImu2gsLzgq2ggGZvBQWs7lbQWVbnjHi19pibOuf3TtzH pt5T35g32Z46feGoeWvIH3hia6Fziuf8MPaNSdYe5pZTs1ZwokrOtq5mjRAaNWuFSKVmrVCJatYK RU3NGiGENWuFg12zVuAqNWuFilOzRkiNmrVCJOraBQJB1y6g2V27gNXt2gVCWblwq1SztlF9dM1a u04gNo31Z09pBLeOnWZ1diy1Y/VjFgAnWTo7lhoxpXg5sOktnUUyzaWzyMddOgtY3tLZUVTgRni1 3vebArd/2NLZlz+VNimyYINee3wdexCrGRrvGP5UwfH9qYKv86cIo+JPFaZKf4pAPH+qQLn+VEEL /SkiYflTBfl+f4qeZ/tThGZqdMJqanQilNSHIF6pPmSlYs3haDpzmcMx9Smd5nDA0fzmVgGf1xy4 cbY57zDLIy4aYmf5Os9pqS44gUdFeLUcOLGJcuDEUJUDRwQvB45IRg4cYaL8AOC18gMLVW30M01N 7sZ+PEY/6RRrGPYlIq/TN0xeG1KT1kE12bXXgIOggqqfA2crTcGJBtUEva00hU04qCbUbaUhBHdQ TWBspSGYLBIXlNJnG1WleKTGjZMb3UE68LDv2q7dt2psh0/NGp8/ff73bhUPv9SIgC113vXnpxgM fxtvP5kWx4/VD+A17c6jqxcPwPdWbwAvsEWReCBDKm7gEb4/IZPnLteA6PG9M3ZvhQ0YhsiYsWva sV9m7J6G7C6wdDtl10iatAmvNmXXXJu0gw/XKbunMbu5sfPscjxuvDZNb48mGh09VfG+/MV26xQE G+3cp53R5WQvRmu7qWP4iwXH9xcLvs5fJIyKv1iYKv1FAvH8xQLl+osFLfQXiYTlLxbk+/1Fep7t LxKaGwFGrGoEGAllRvHUqRnFhYo1abe1zdlfpDW6R38Rj/JbtTLRNetd1Jrz1O9szMzuWna2bTk7 6ApOtKSx1dtBV9iESxrbuh10hOAuaWwZO+gIJlvS2CrtoNuoqmtlnBny4P2pVgZOI1Y0xkOtzHJq nwYNbbo6WLMtMW3Ddcdv49mxXT+ZgTMVv+AEtTKEr6yVQYxOrQwx1dbKIIhZK0NQdq0MoaW1MkjC q5UhZEWtDD7Pr5VBNLdWBrGqtTJIKOtxGvR6nAb+VPwB7PmTrs5xmk66Go/e0NVwF+XXQkwTLtla 0N/s/g0bO97UM8SJups6zalnyCbsbupqp54Bgtvd1LGmnuGUPFF3U6c29Wyh0upumqdLDYN5Kmfd LNhs1kkK3oVTQDdbn3vJ3hpWtXfBiaaeKVZ7FzbhqMzKam9CcEdlcqq9CSYbSKVV7b1R1W5RzkOw rsMlZfstynhq5hi6/cbB7bQPb4qHX4JNJpsULunANPMnLPezYeY7jDDfYVTzHUac7zDV+Q7DzncY Xr5D1C5EeLV8x0270Ds6xgcz+mHGHeP7XiE87bB0wu57hbbT9y4eDM0qJt65fPLzYsopCpYpj5EX B0GcxLZCvJ5thWwy2woZ6mwrQDBtK0BybKsxyuIggFcrboucOMh1J6cti+gOeQo8eprNfyMaeRsr a9MlY9637CJR8KymmVGiX3CimSOArynRJ4RGiX4hUinRL1SiEv1CUVOiTwhhiX7hYJfoF7hKiX6h 4pToE1KjRL8QCUr0iYBdok9oftBnmpWDPtMsC/pMSgPvN6r6oA8uZU3nK9jkeL6CTX5jeqZrgg1b xMdfipUC2Ddc39bZlAOrPh9xkvp8xOvV5wObXn0+kmnW5yMftz4fsLz6/CTcTZf0dtOlm910/7D6 fLiAFwPGu1LT1Fh4/GLABDOzp0L5yfZNGzhLhgCXBAYM4vtXDdGxfghR8e56BRR6Ij/+64+Lrf7X zwO8xpcCv/sXYJvurA8kCxeyWx6geGIos08wFfJjApX/5T8/b/pzf4gXgHl4I7tlCs8nMBu+fdlb ZedTEvy7pBSwoAqF58fpV4T8OH7pb/4BuOLdUnXiGo5cVxL6Sx6+UOAh/wYeLH/7Ct4doFn08IsH T3bADrf+BKDhCdQfft9P8+U3/jTj3zs9wNcrbvdbXz+T7XC72KpvRlcuuB1w+5EshIc/WLYrqlfL Cy1Up3qyy/1Gr2cf6fNTnrsu98dIH55aM/pDFHw9DeaNXWrZ5nW8l7dtPO9Mzk3s2EtxfA9mGcNY KDi+sVDwWsYCsWkZC4VMz1gofDxjgbAcY4GAsuLMWWnA6kb1wc18wedtYH3rLpOH/dgIkqzJsYoz CScoziR8ZXEmYnSKM4mptjgTQcziTIKyizMJLS3ORBJecSYhK4oz8Xl+cSai2e35gNVtzwdCmePi tIozVyrecJQ0XYajWNu2p+EoFndXvFHwEZp1kJl3l16lPLaigZ09Z9pzwUla+RCv18qHbLJWPmSo a+UDBLOVD5CcVr5eZPESXm3U343F+/+n4GPs/9n8889j/7vp84/p2+++dp8/Df/zf8Ad/vXHX75N /+PLPP/m/wGE3IzC33EQAA== --=-=-=-- From nobody Fri Aug 18 23:03:53 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSHW31GtKz4rF3h for ; Fri, 18 Aug 2023 23:04:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSHW15DTnz4lY3 for ; Fri, 18 Aug 2023 23:04:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=jcNDYR1T; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692399851; bh=azT+f4evV9xwf+4GNihp8vcHoD8JhdezvhsOvdDCs84=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=jcNDYR1TgTAB3M5NfvDOJ/J94CusKpdXK9G/uYXrtgHxn+Sx9WhPv5ljj6PCzYYFKvpDS9gELpiQh4U+irM4PFkNw0mg0jajvB+LcnpBkefthOyRveNy8iKnhw7GkxfxAnkxJbAMKS1qjbuPvsxLnEXE96cOncqFgaSxcVK7GEv9Gqw4ldXlkyw2otssu58K+99Qu7K/KH5JerTzTWBXMsGLgBQ+bazIKEbBcLFG5rJD7APOsqotXH5XJGZ0BVhYZ9DvisHE6r1cazFELKndnh3WV4/jhU3WCS1jgvuitFaeCILn9qhQYbGynkNDYYOPEgkLWkM+nDpAtfhaCFd1Vg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692399851; bh=D7P2eWMYKnKfVYKXMjO+Ovr/8t6FHe/ANA+6jiwzlDv=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=fwsCQtGLIfDBn/gT5R0SAEU56eIbqqMFP4LFgwh+S8/rnOf6N2jNUjWiFzg2KC/YIbf5xkcw0iRgHAhevXzsd7v4GCWtL7yFfBzriV4FvUZSYLfi2uZN46jpYI0WScq08eG1JnHSiMdlgk3Rcx4zbeDQu5yxpoAesvoVnoaaquogBpVYiPnqPFcwekYTPCHics9oWk1nJdfbNEYqdIfpdwgth03hrPsFSLiOY+tA+PI0/N9vOoq9Tz2gcnLAKk48HgImG14xK1aVc/6f6JVzGVR4OBkkZkPjZwPAJ632WwqnO/i+k5SR2vxA9NRM7f5QUwlJBeV6CVhkGu/HUIWzUg== X-YMail-OSG: LTtymzgVM1lSihlYVupuLFWgBFo_dROjZUvYIe7WeykYKATCd6C7pouJoVqwsGu 8S7LZuxOb31EsOlgeS7TJ3E9i8pr2hFW1I5QFMFv7mhLE60u0tjGI2SY4.AATNEwgHTFizhD9Jjk kCx9kD2ydSf._B7A_z38GdHVr1b8NWYLEFihif2Okqjz1jg4umhgJg7h8Xh9Dvte9TYF42P7_Qhl wOAP9Dqr3jdBmlMiiH_6E_gHsx7NYwmj8L_c3_8pkPG5VJ.Zwq_.UcGNUlQizfJYt6tyCBFll6.T tseW.SrU4UvxDP3OIiVY45GG1H6wl1mPWfwYI1dGmqfUMgsUcF0MZetEv4omOH6QBibKfPmqJ95e 3j3rylvFYogKZTSQGdSvZGLtEYDzKclFRJJaJtCKnVHaen_Phiit8dzvjXrjokxuUt1VwY6Tesyf YEgQKpVEzsgeWQFsHRHSVUVDbs6n93fbMd5fRQ61qWSbVS0Hqay.FGwe5LHcQ6E1jNXlL41Xh19e WYO.l4q0yD.7T4y5T3uSzmapz4IpgO5UmEx8WO9vSUGOmZQRv.KY5Ao3o0nYwGLn3bEK4yj6KoWo gh3UkjOTkA8PZ29HbxgobDocjaFtNy5KtsVTr0PWmuW5cay9xAf1_zqqiUjeepsfeniw2TTL5cPr Q7Dm7jrkYjFk2p_BSc1KIDdJlZoMZDRpAlch_QhWwWMRuzrCNXYVZTJljdZz_JlEne8grzXbfUXo wkUamzwM7AtCw3PNLvwDHzsFRtaKHyXLB4IAedSojxiZwVKcWj_mrbWFqWP78qz.CMzObdO.MGL6 VRImiRb0WhFZYHhsIko6Tn109X89qHUiFAIpqKpm470uYHL4V5tDmRaRDcVzhn0lCDpGonPpVqWo kwk1mEMVViYu.bwQw6rzwxucZJuD6RbYZtQOq0_UNUWIdLcIQzlnBuSBkzY9znML0FXduyTqvdge sNAcArnfGDR8vRmUGfmNEMAW6l6aPA6fubdjtWD2v.IYhVRGVtxM6flzkLNtpMx3n3AI4Yf4QxIF QEw15cTlUxC.6tTy223iQglBdw1iHV_DIzyQzwpjZU5NHRLyekZuBDOXqgAEocdnkVjczskuNkyG 7MV8hvE8wEEhS.xzBlNB2Loc8tUGrEaoMdbmx2TbAev3jXt8lKK.vBgN8hOryF.PPVtE2L8cgh8f igdVpWSJ_Nzrw6udmWd9MWK7HDfcojh73AGEwe2SHmSICC7u1PrKtoRMGn7o7IFfrJjrL2_we3MK GzZApIP1sqp.qmLBgIE5t5jPYS2gFAIAS.DiFQmm5znU2araNCRTj035joFDTxDbmz5rxEI3Dt07 G3OwBzt.G7SvPTc2qsbyDckUqDzr__6813l.Vl_d_DctIn9pp_1jb8Bj9b9QobdXoxryEKEkHJQ_ Gr3VJkjmeAF20_oldfAS8QZgPrBuzaE_js_.NASbJbxawixUXahbXbF_O5FctqaFBluDELMaEQvb bbuCjzQBqsQpIdhqvCAYQmOIcT0WrFKcJNvafZ6OKfWXJLzzdU2vbmonPZlyliWdnXUWEt.mYAv_ uJNnCE.1ltbCv_0z61XoP0PkmJ6BvXdSGiJBGs16js5V7R.NzMHckGEnpkwBr6_SImGr1X_ppjX5 db.v0wtp2Fti7st_GJAA1vAeIT9cQZgQQAHme9fbZkrDnbR7ivZ5WqqyvdCP_JXG7ppuo6qcneLM xlbqFqVfrXghW_XvojJKK9X13Rf5M1uviAv7b4_pfYyC3DBjSPDFEpqVYUF2mp1rWf0Ez_vXBiO_ hca1fEg8BbYWMSCcIcHCeiAGTNUVWXdu1_AyAf_NWkQyg3vOTadBoV0ZTps5S80V_X3T5GZXSqoO FZ1sMBN3Kod3ZAdA.6SCG.ojvLw3LBoduF9YncnI7JdnuapXuIPtHOSnLIzglfb1fAoVos2sevHZ wljEaTEwNiEoc.mrelpnmyYrg.id0PVPkigQ4ZfUnlLzz4AaWTY4sOKsXurbbOZVRIEx5RyTHg8r wuSbEQutRIK6S3li7UTYt0InLS_ldNtFBL09lUpCWErnNvVfPYlBL.WAYJAKxvX6TU5Ik1V_coq9 9BZMrAhXLqQjeC4dQBjQkjDSJkzqNfmuEAgIDk30LTVp5h.CcOFLl2wjPEJz6M7YWvcsj_ZMqxdc s9k7s0ql5Pdz2l1w2yd5ZTTwTK.q98NZ9ebpL4RDf0S_UyXjEtmLWfzJ52Un01.kKwyVF3ga4_qZ 0KKT.ptXOw0yKMQtKvqB50Rlc_Vjw23P2cj9Q.vooaDbRWvM1TTlSWnrJ9xayW664.uPJ9Lg9p72 zFmJaFQ-- X-Sonic-MF: X-Sonic-ID: 2acbb994-e1b8-43a1-a954-66e54631765b Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Fri, 18 Aug 2023 23:04:11 +0000 Received: by hermes--production-ne1-7b767b77cc-7tm2h (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f1fb688c6a0f2a647cf8a45fb25b917f; Fri, 18 Aug 2023 23:04:05 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14 Message-Id: <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> Date: Fri, 18 Aug 2023 16:03:53 -0700 Cc: kevin.bowling@kev009.com, Cy Schubert To: des@freebsd.org, Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: <59FCB309-4A55-4924-98C4-7ACCA70FD299.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSHW15DTnz4lY3 I believe the below quoted messages were reports of deadlocks based on after the following 2 MFV being in place in their environments: Thu, 10 Aug 2023 . . . =E2=80=A2 git: cd25b0f740f8 - main - zfs: cherry-pick fix from = openzfs Martin Matuska=20 =E2=80=A2 git: 28d2e3b5dedf - main - zfs: cherry-pick fix from = openzfs Martin Matuska Kevin Bowling wrote on Date: Fri, 11 Aug 2023 03:32:51 UTC : > Spoke too soon still seeing zfs lockups under heavy poudriere workload > after the MFVs. Regression time matches what has been reported here. > . . . > > In message > > > om> > > , Kevin Bowling writes: > > > The two MFVs on head have improved/fixed stability with poudriere = for > > > me 48 core bare metal. Cy Schubert wrote on Date: Sat, 12 Aug 2023 04:55:40 UTC : > The poudriere build machine building amd64 packages also panicked. But = with: >=20 > Dumping 2577 out of 8122 = MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91 > % >=20 > __curthread () at /opt/src/git-src/sys/amd64/include/pcpu_aux.h:59 > 59 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (offsetof(struct=20 > pcpu > , > (kgdb) #0 __curthread () at = /opt/src/git-src/sys/amd64/include/pcpu_aux.h:5 > 9 > #1 doadump (textdump=3Dtextdump@entry=3D1) > at /opt/src/git-src/sys/kern/kern_shutdown.c:407 > #2 0xffffffff806c10e0 in kern_reboot (howto=3D260) > at /opt/src/git-src/sys/kern/kern_shutdown.c:528 > #3 0xffffffff806c15df in vpanic ( > fmt=3D0xffffffff80b6c5f5 "%s: possible deadlock detected for %p (%s),=20= > blocked > for %d ticks\n", ap=3Dap@entry=3D0xfffffe008e698e90) > at /opt/src/git-src/sys/kern/kern_shutdown.c:972 > #4 0xffffffff806c1383 in panic (fmt=3D) > at /opt/src/git-src/sys/kern/kern_shutdown.c:896 > #5 0xffffffff8064a5ea in deadlkres () > at /opt/src/git-src/sys/kern/kern_clock.c:201 > #6 0xffffffff80677632 in fork_exit (callout=3D0xffffffff8064a2c0 = , > arg=3D0x0, frame=3D0xfffffe008e698f40) > at /opt/src/git-src/sys/kern/kern_fork.c:1162 > #7 > (kgdb) >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Aug 19 00:10:59 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSK0B4yNYz4fw8g; Sat, 19 Aug 2023 00:11:06 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSK0B4Tpjz4svm; Sat, 19 Aug 2023 00:11:06 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692403866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=PhHInTX0oaR8Loj6VGOfRH0eHjN/iSY5nx1cjy6ANNg=; b=RCgExbtPqeuz1tpbhoo4AzNjdt78pVX0bx0UEyKBPXVqVw2vVeVlky5KSCgaLS0B7Njqf9 MwotHErg6GFjgsQF3k98Kk+NSH+J87hefzM6Uhk31PR//IRQzJRMHTxTBKQm4As0ZUf/6v 8iwU3xBUC6Pb/yr11M2oPqobw2DrsUzJJKp192OfSjwuNgCDm4P1efD5DxbKX/syZpdIEa GXtIQjK8xeiqT+1UHxMrFa+WZNsJz1wQn9qToMGW4QsJo3mV0dluN8XpW33HhRThCFaB7R Ab56itu1Y52lMQxh66dsG0J46xymOt5v1RFyRuBv/kM2649r36cUVH14Vp62PQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692403866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=PhHInTX0oaR8Loj6VGOfRH0eHjN/iSY5nx1cjy6ANNg=; b=CHNYRmQjVWU4ra6pO+nxIjhKfri+wKXjXe3+9Osy+YmD3EA9kl8A2Edd27uBm5qibnUopN woLee3XpN1P2YxyHBvJKMnL26HqN854ceTvhfPBMas91Xaol41oeOkg09WsFs4YJrqjcIU oqM01r9lG/ZYNAgdEyh12WpxLrwQ4M9+3xK+KqzaYKnqIwBmMveTeP79UYzaLtu7KsPjXZ 4AJI3FCJ2wEJDIZJuYGa2g3O03iVGyjR6CrxpQp4UVKvVQoF++LyaJk/PzwNkg+e21ZD/0 lqEF0owOXdaal39TRmqqfaeY6+vIIgD0e24wM3vtDnq++Fv5loTUUfLJeHWjIQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692403866; a=rsa-sha256; cv=none; b=pg+qJkczX/53WmhJ6qUKYzfgqjzJNk/Fa2jjsbjmj8zRbgOustaS4JqwodynDXCRdeLpri cOaEDLc7yPxZeeTHomYZREV/VROuZS3lIhjaWm0HLdSvtCLwaLBoZpcASW+s+MFhGc8uCd O9f9uK87uDpVbO7yMCuhnrS1wJoUBQytEE0WrFevLEoRX4L9Hk+FonChe+Ng3P4Cd9tb0c sHGIyIvszRtCWr8NAOaD9OLa3B+zB0eopHtgz7hLxr4dFWyYxOgM8YVxh3hR55B3eQLVWC qJs+ky/8BAb1Frk0UheoP6gsMzdK+S6GlU8nJNui5gX+izNO/zr+s0pdgdQ4Hg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id CF3ED12185; Sat, 19 Aug 2023 00:11:05 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Sat, 19 Aug 2023 00:10:59 +0000 From: Glen Barber To: freebsd-current@freebsd.org, freebsd-arch@freebsd.org, freebsd-stable@freebsd.org Cc: FreeBSD Release Engineering Team Subject: Possible regression in main causing poor performance Message-ID: <20230819001059.GH1219@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="f/XZMZU9ST6S8MjH" Content-Disposition: inline --f/XZMZU9ST6S8MjH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline I am somewhat inclined to look in the direction of ZFS here, as two things changed: 1) the build machine in question was recently (as in a week and a half ago) upgraded to the tip of main in order to ease the transition from this machine from building 14.x to building 15.x; 2) there is the recent addition of building ZFS-backed virtual machine and cloud images. Here is a bit of the history of the build machines: - the second and first build 13.x and 14.x, respectively. The third is (at the time it was purchased) newer than the other two, and significantly fast when it comes to weekly snapshots. I'll refer to these machines as such: first, second, and third. - The third machine will be eventually used to build 14.0-RELEASE. The will first continue on main, where it will build 15.0-CURRENT snapshots. The second will continue tracking 13-STABLE. The first machine runs: # uname -a FreeBSD releng1.nyi.freebsd.org 14.0-CURRENT FreeBSD 14.0-CURRENT \ amd64 1400093 #5 main-n264224-c84617e87a70: Wed Jul 19 19:10:38 UTC 2023 Last week's snapshot builds were completed in a reasonable amount of time: root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c ./builds-14.conf ; echo ^G 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/logs 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/chroots 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/release 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/ports 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/doc 20230811-00:03:13 INFO: Checking out https://git.FreeBSD.org//src.git (main) to /releng/scripts-snapshot/release [...] 20230811-15:11:13 INFO: Staging for ftp: 14-i386-GENERIC-snap 20230811-16:27:28 INFO: Staging for ftp: 14-amd64-GENERIC-snap 20230811-16:33:43 INFO: Staging for ftp: 14-aarch64-GENERIC-snap Overall, 17 hours, including the time to upload EC2, Vagrant, and GCE. With no changes to the system, no stale ZFS datasets laying around from last week (everything is a pristine environment, etc.), this week's builds are taking forever: root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c ./builds-14.conf ; echo ^G 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/logs 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/chroots 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/release 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/ports 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/doc 20230818-00:15:46 INFO: Checking out https://git.FreeBSD.org//src.git (main) to /releng/scripts-snapshot/release [...] 20230818-18:46:22 INFO: Staging for ftp: 14-aarch64-ROCKPRO64-snap 20230818-20:41:02 INFO: Staging for ftp: 14-riscv64-GENERIC-snap 20230818-22:54:49 INFO: Staging for ftp: 14-amd64-GENERIC-snap Note, it is just about 4 minutes past 00:00 UTC as of this writing, so we are about to cross well over the 24-hour mark, and cloud provider images have not yet even started. I am inclined to do two things: 1) Immediately run a subsequent snapshot build to see if it takes longer than 24 hours (my gut tells me it will); 2) Reboot the machine with no other changes, and immediately run this week's snapshot builds again (not to be public to avoid confusion). Some interactive commands are significantly slower, such as systat, vmstat, even top. Creating a new window in tmux is also noticeably slow. Is there a third option I am overlooking in trying to identify the drastic cause of this? Thank you in advance. Glen --f/XZMZU9ST6S8MjH Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmTgCI4ACgkQAxRYpUeP 4pOKtg/8CQioy5hX+3kT5uGpdv1IokbSJ+Wg7vhaB99RuG92RbtZNufmgh1bzbuj GyBD9vkwfNExjhOYRnvPRoDpwZC4cufxIANKi0Cwl6DKxTCrMOF3UGM3hM2dY4lm 7bniYolDgzm0v0GzAhgafi0q5Znsm4o4dslm0iUdLqj6ISH2fFI3FJoMQ+moDt/c vCNo8wz0BgCLA3BVILHzwdjoecksONLyoa9K8xYE/YfI2uXt/RrRzcHdw5qGKyKA eKpV6MBZF3ITawIt8rPUWp0/WGPun+KBW/uD0QumDW1VCMH+tLpyuB6vh4EtCOyj KOnTp1zqUBoLN3dxnwi58wDOy+WA++QAHcRB80UmuCYgU1pGYAO0aox77c88prb2 VGpAf81TnLJDiB+Ibphyxx0ju8pujQtTbywu1wmFDur4dtmfrv6ZZTTWwkb6lucD TygH4y2fQqzgD9Qcx/2nOH7Ax0KPN7/W4xi+O0537PcNAzJP2UM+gaXnzmn1/HTn aRkmLy3oOTlL3mkSi9R12t0B6TpmVbRL8cUrBMtA4et/n4v+7eMXsKgVn4JWYqv+ vB02mHrZFSDsb6jV0z+T/F8LHm5NMmAGohfpHCXZCtJ6VI5Y/350Faslj5bznVyA 5DyCXizNyxJgCAvagCRX3G6Rg/B5HytlHUnK4s9qYSDMHRNin80= =WC7c -----END PGP SIGNATURE----- --f/XZMZU9ST6S8MjH-- From nobody Sat Aug 19 01:54:14 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSMHG1Xstz4g3s8; Sat, 19 Aug 2023 01:54:18 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSMHG1D4sz3NHh; Sat, 19 Aug 2023 01:54:18 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692410058; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=OiDKgivR1NuuuA7k1Xo02ZoMXn8fEhFykU73J8tAyQM=; b=F4h/f/6sA7VG6d7OabC+4wK9EfpWrMeqz8lrb9xXlEJdVqXsnwlsdiuc4dwMdg3HWpWlrb owRniuz/3xnPuzY+w7BV1OQ3qYUkcqri85RVVjlPx2ZDNJz1JNQ31vHYP9yJzfAk2OkBLz OcIqWFQY7kS2oxwGFGY02QVLzgB/K1OCJmvxIiS44C8HJ2+EaLHZQQdEMWkZH07UC4DwCI a0rw+b85cMoHKHwQypxTcy5bi6+HfJHshSlFl/hfP6iQLXnQFPQdQCHTeOb1f2q5kr/ak+ tf6GqDSxia31K6/FhpfpjSjYwuYlYXG2xzCKOkh86V5I8JlHZ84/XXGJk8zeIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692410058; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=OiDKgivR1NuuuA7k1Xo02ZoMXn8fEhFykU73J8tAyQM=; b=mQRaBE7rE7KAssxKw915W+1YR69ADkvZ0dXDu2VPUX5p/xhQZQYhpPaV0hQtyM0kJb6Z7G LRUIEYx2EmZePgKp0R8pZsCmPDWcGNn6/XnKcEekVz+TRRt1f7XxtyjEAITl3qYp4oN8Tt 1TNe2P/QQ53TXvz78Fj3pdhvdPEQmOjTZfX/nhHr5aDMDjjog7CRWyVv2tdmg2UG2Jvw+e kwKC721sZNxImZ/Dzr+7s8nXX1aEegl76VsL1+TC0mRTkdXI8XDwd9Afwc6vKw/yhTt98v tdG1kFXe9WDECajsjVC63C+kg1dkXdz+aLlMIU0S4/idbUdxSOYnoRhSiuDRkQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692410058; a=rsa-sha256; cv=none; b=F4zD4DB189w8eq36DZhV9dD4qEKaauMzPIZLuGDcGfY/krZtDgH0vfVfE7U22iSCTNuo4E 0Q7g8na39oZEI+uB8s57ZDids2BO597geMquoXH9q8ZkuGbQNKDa3mInAcwgQKVYJO1xnq pl5OYil8SzYW5F0S2fxPhUWWOZN1hOCtQLpQAvLNzy3iD2usC5QE7Nmmm0e4wvyPA4DSmY aGXcl4oi6uK0biFadqHPj5+MPxO43ATyDEO37yWOnMwHVhQG5r/8eAf31azL6x+4bhBHm+ rRQWc4XwunCy4+zlM//AdCcIaaqcr6cbiqLJyHHPYNJvHVS0AELQFp52LkelKw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 5451912362; Sat, 19 Aug 2023 01:54:17 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Sat, 19 Aug 2023 01:54:14 +0000 From: Glen Barber To: freebsd-snapshots@freebsd.org, freebsd-current@freebsd.org, freebsd-stable@freebsd.org Cc: FreeBSD Release Engineering Team Subject: New FreeBSD snapshots available: main (ALPHA2) (20230819 77013f29d048) Message-ID: <20230819015414.GM52318@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 New FreeBSD development branch installation ISOs and virtual machine disk images have been uploaded to the FreeBSD Project mirrors. As with any development branch, the installation snapshots are not intended for use on production systems. We do, however, encourage testing on non-production systems as much as possible. Please also consider installing the sysutils/panicmail port, which can help in providing FreeBSD developers the necessary information regarding system crashes. Checksums for the installation ISOs and the VM disk images follow at the end of this email. === Installation ISOs === Installation images are available for: o 14.0-ALPHA2 amd64 GENERIC o 14.0-ALPHA2 i386 GENERIC o 14.0-ALPHA2 powerpc GENERIC o 14.0-ALPHA2 powerpc64 GENERIC64 o 14.0-ALPHA2 powerpc64le GENERIC64LE o 14.0-ALPHA2 powerpcspe MPC85XXSPE o 14.0-ALPHA2 armv6 RPI-B o 14.0-ALPHA2 armv7 GENERICSD o 14.0-ALPHA2 aarch64 GENERIC o 14.0-ALPHA2 aarch64 RPI o 14.0-ALPHA2 aarch64 PINE64 o 14.0-ALPHA2 aarch64 PINE64-LTS o 14.0-ALPHA2 aarch64 PINEBOOK o 14.0-ALPHA2 aarch64 ROCK64 o 14.0-ALPHA2 aarch64 ROCKPRO64 o 14.0-ALPHA2 riscv64 GENERIC o 14.0-ALPHA2 riscv64 GENERICSD Note regarding arm/armv{6,7} images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root, which it is strongly recommended to change the password for both users after gaining access to the system. Snapshots may be downloaded from the corresponding architecture directory from: https://download.freebsd.org/snapshots/ISO-IMAGES/ Please be patient if your local mirror has not yet caught up with the changes. Problems, bug reports, or regression reports should be reported through the Bugzilla PR system or the appropriate mailing list such as -current@ or -stable@ . === Virtual Machine Disk Images === VM disk images are available for the following architectures: o 14.0-ALPHA2 amd64 o 14.0-ALPHA2 i386 o 14.0-ALPHA2 aarch64 o 14.0-ALPHA2 riscv64 o 14.0-ALPHA2 amd64 BASIC-CI Disk images may be downloaded from the following URL (or any of the FreeBSD Project mirrors): https://download.freebsd.org/snapshots/VM-IMAGES/ Images are available in the following disk image formats: ~ RAW ~ QCOW2 (qemu) ~ VMDK (qemu, VirtualBox, VMWare) ~ VHD (qemu, xen) The partition layout is: ~ 512k - freebsd-boot GPT partition type (bootfs GPT label) ~ 1GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 24GB - freebsd-ufs GPT partition type (rootfs GPT label) Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. This is provided by the emulators/qemu port. To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt -bios /usr/local/share/qemu/edk2-aarch64-code.fd -serial telnet::4444,server -nographic -drive if=none,file=VMDISK,id=hd0 -device virtio-blk-device,drive=hd0 -device virtio-net-device,netdev=net0 -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. BASIC-CI images can be found at: https://download.freebsd.org/snapshots/VM-IMAGES/ === Amazon EC2 AMI Images === FreeBSD amd64 and aarch64 EC2 AMIs are available for both UFS and ZFS. The AMI IDs can be retreived from the Systems Manager Parameter Store in each region using the keys: /aws/service/freebsd/amd64/base/ufs/14.0/ALPHA2 /aws/service/freebsd/arm64/base/ufs/14.0/ALPHA2 /aws/service/freebsd/amd64/base/zfs/14.0/ALPHA2 /aws/service/freebsd/arm64/base/zfs/14.0/ALPHA2 === Vagrant Images === FreeBSD/amd64 images are available on the Hashicorp Atlas site for the VMWare Desktop and VirtualBox providers, and can be installed by running: % vagrant init freebsd/FreeBSD-14.0-ALPHA2 % vagrant up == ISO CHECKSUMS == o 14.0-ALPHA2 amd64 GENERIC: SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-bootonly.iso) = 02b4a7ba3a9520031607e6ebcbdf6cd142044aa1af40ad452c9d9793c5e70ebad94b13e679f5cb0aaa473e20f9da1f63b335b0cd25323aa03b14bd539b565ee1 SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-bootonly.iso.xz) = 966041afcb3df8cc35063592d67ad146453783941bff80fa019d42df6e624dca2a8763940f20a7cc3857bf708e6f76e1420ed18af4a1d287cec004bdd31893ee SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-disc1.iso) = 76f8ff46e679d4920cfe67179a587abdfd323e83c3e3f79f39678f6a956b888cf4a2a1962f9a0549a1fb481912ef41fb666bd518dc1775e90488c31b841d6aca SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-disc1.iso.xz) = c61970abd7f2b7c0a81d5bf702e3cc65129843e3624b1a93a1476ffe535adb8dbb13378b3b049d2e742e47383c3f6bb3b2b0678977ee11873c0b50414070ea1e SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-memstick.img) = 57cd164750830d7679023e2ee99084f496f9a45527f488704c464d6581cd682b453d178194b355ea6529b29b5cb902368adbf872adfeb8920c2973d685a5034b SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-memstick.img.xz) = e213fade4c3573a7b7b2bc8fdf08cfac02611d7d1d0454e0d8a9971380ee8849c6b64ea38676762bf4cdb74b6cd282a91b68ba85677c528d9ef09a72fa274374 SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-mini-memstick.img) = 46011485ebba1b9e3e2d6e99e1a5254ae3ac08844f4e4b888d207e8c7b67de1e65709747e4cbd7c2e16880df7cff41d0e8faacc5bdafdff14a21d4860e938e04 SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-mini-memstick.img.xz) = 6b12574ae85ba543f7f8e4df3c48ceb3eb655592c8bdc61f98a0619b9df40a14910c917d88361da007c2d2df35c14a0a58cc77ef5178648980b1b4d5cfb4e632 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-bootonly.iso) = 7e803a3827d4e1efdcbfa3e195670ef1d8c7b61972835a67730f24d27b2bfddf SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-bootonly.iso.xz) = a4f4bfc41d1b9ab5bc9d05105c4e0f847096bcc6a1179c1cbaf5a28d61e84d49 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-disc1.iso) = 1c841c765a7b40c94006615ffa9e121ae46929f88109f4e3d8fe21c0d55e6729 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-disc1.iso.xz) = 903c4fac59ea40f53ef016a0afacc05821a1f5ce9c8953d95a88f39a5814d3a3 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-memstick.img) = 080f3fe1c939f3743ab01f8ccd73e6a0d64cb7b069b356c94253981463df9440 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-memstick.img.xz) = 1cbbfcb4439092f014fcfc83b04385448878ee439a7d3d6947c8203d957b6928 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-mini-memstick.img) = 36ac6dece7c7885868c514e6ec40051077ffca369f9ce99eae67dcfa95a984d6 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841-mini-memstick.img.xz) = 2756fabf7513c7ac94b99897f22cac8ff1ce3ff69fe6849ca029caf6e502f2a4 o 14.0-ALPHA2 i386 GENERIC: SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-bootonly.iso) = 3f327f1bc4878848a8c59d489fc8755859e14c0568328ddcaba9481fa935f8cde00ea8eedd7809fe71954df70e556794426170fb971f06fa4f9a4cf3f242d04f SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-bootonly.iso.xz) = 4635b8e961697d3383b74061859068c83363e25cd1c52ceb0ccaa3420ef0fe3a25120083999553697c95656aa8a02a5ff23f369add71913429d9947cb70c2e93 SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-disc1.iso) = 50c5d691bc0e75478aeb09583e82670c71fbc108a1e29a16269b7d05d692596f12039e9c52775b3eb2deadfcfca2f8efd8500a50df9669cc2638bf131fc9d8c4 SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-disc1.iso.xz) = 188ae1a12122f3b465302803b3b70d8e983f083ca19595bc2f7dba227575bbec652b7d5e8e80810cb30dbeacfd698cf5a64e8255e37b5dad9f17f951d04014d9 SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-memstick.img) = 92f94f2a85c5b6a949a125ae01ca97094f548db6758474fde157398ea514e92f33245fd11afc57137501efa66e75ad66cc2ea1fc966c50f0d5c724c57d4daf27 SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-memstick.img.xz) = 748343cec3f4927767f33a49301653bde57dd81b1e94ec3f2496ea092e592ade229b3c4a0e66c049d614a096de6658a44ed5891ca4102cbba85854874d74d9e1 SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-mini-memstick.img) = dd8bb4d92c8643facfc82a066173a9d43d7f2900ac36f29490bc0eb09c789bfa1644befe9f949ed6d886ef5eafcad7fbdd9335195785e201e75836c4d6f79348 SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-mini-memstick.img.xz) = 4e389a859e7bf2b306c5c7a826976632d7448154c8c5da2e2a133e647be0deda678fb6f979611f33a8d45a08dd8b81ed4b66bfb7efc304ac19cbaf58d571f560 SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-bootonly.iso) = bab04ca299555a647d2ddc9480137cdffb6855b4a841cd595216f32456884a01 SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-bootonly.iso.xz) = c21780d48988c11bf121564b30267549ef9f29f290e745701ec9c823a8396635 SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-disc1.iso) = ad39bc1c78ac407b59e3025c47330e38aa9df5097561e098cff88d48d8ee333a SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-disc1.iso.xz) = 3dcdb15ecdf92327ebcc96e1de0a742f3bf697fc69d91956e291e58dea96ccc7 SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-memstick.img) = 1abac8845d6387f5b56e0926c75668fc65b6a5431af3ef68b0c333c458fa64ef SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-memstick.img.xz) = 3d230292a1f5ef9b3ca37212c070f77fb7333c41e0624e4dad203bcc08e9f86d SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-mini-memstick.img) = d86614bfd0bf42e69314ca6548723ae06bccf0df8ae12106693b2b14d83941fd SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841-mini-memstick.img.xz) = c42530736db3c52d9088a2e003adc76e205f96dba853bf8a215c64d8b5142889 o 14.0-ALPHA2 powerpc GENERIC: SHA512 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-bootonly.iso) = 4e66aaaa5c04ca44d20c252ca24342ed13b6b70a1597de38a8259365d156c719bd58f00c66cdf993cf08d3d35e32708d6f0b26393e3b3e9e0e443be6f663d645 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-bootonly.iso.xz) = 5ee0eed782262bdabb125597c83cc7b75edba3e561317b56f894841d0fddeb354d65a80bec777e8dfd280c028908e6bac2194ce5d9e9cdc66b9f45643514af62 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-disc1.iso) = 5ec97bc68aa18c9025b93f1a7bf39a2202deb369fb3db9ba26b81ea8a59ef9d08226f020a9cdc9965ff8105d857da42c566a8cd37d95475a297a7e512f4a33d3 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-disc1.iso.xz) = 894f38d787f1cbaac011bf1a07e5e3a4f4eee6913b7994c92f33db5dc3ded473dab0a20170d8d5a32aae0249a90b80d28faadaa77e790e18d28f248b365961e8 SHA256 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-bootonly.iso) = 3cf4a67508f06655a1181756bb0462893bb00ba0539d34f4736bc2cbd97f0bac SHA256 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-bootonly.iso.xz) = c9acc34f9f76c0c597a3b48c1e6ee755ac10ff23bc9fc14997d4812c4d819f3c SHA256 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-disc1.iso) = 2e9e66c3ea1e29cd6d1324a7fee73378a302885a67ac938e9273a70f3fa4196c SHA256 (FreeBSD-14.0-ALPHA2-powerpc-20230818-77013f29d048-264841-disc1.iso.xz) = 46fc17932318730d3cbd3ee8b540b5174387ecdd26bf0df96d105b889002d542 o 14.0-ALPHA2 powerpc64 GENERIC64: SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-bootonly.iso) = 5ad6e4b256dff9531c0bbbb9d73044bbbb55113103bedf73db79f4b7bc45b1e5e07aabe504e74b8b4ffbd8b496cf7e006a26dbbd4ecd3ce90a650cb375b0bf00 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-bootonly.iso.xz) = e365284316e441dc957dd147e4a7bc1d967da83a7fd4407f599d4f41b5ce89adfb19c4df3217c03b1a6a2b5182e0feddf21cc5036ab3001583a4099eaf554e7d SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-disc1.iso) = 7018ea753ad4809d1e43bf99bdf3c490fb42abf45ba262dcbe86ef6ec7b1dd1d081de845d5a47185bff974c851ced1243b2c4f510183613db42f92d5cd30aa99 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-disc1.iso.xz) = e6ee34d78bf4ce10d735eca5a7c9a3ea72a51544a6949234557e5e3789ce25ca339288a649fd22c36481ad23536d4cd275e26aaf86c7d5b5a987cad49bf7d48f SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-bootonly.iso) = 94127d283e5ee867ec456b09e553bff29f12526676c246ffd7fd3217429d5a8c SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-bootonly.iso.xz) = 0202c57c6187f5b4240f938d38d6f21913bdb30f1e08f8b8530340fdc0de209f SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-disc1.iso) = cb5910bbd3a65d2ca7ffd52f88be455726371a3e7d7f1da50da41e5254ce13cd SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64-20230818-77013f29d048-264841-disc1.iso.xz) = b5398f73d0be6d3b0f642cd4c2330a86ad14962a5c94eefba05ecfc1365e21d4 o 14.0-ALPHA2 powerpc64le GENERIC64LE: SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-bootonly.iso) = 3579beb2361d52469f3245f7af4a1890e0a49368cbd16c1c70989c2745d9a624b6beee0c3f02f72e9d66d61e7b6bcbc385e236a96673328daca9552e75fe802d SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-bootonly.iso.xz) = cb6ddda3fde1d28c01e343961d105e26c58c48c6e5a21ff95e9329195f81df90192158581a8bee3b4bd0dc80fb05c68431ac5cedb11a1605ffdecc255a311e2f SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-disc1.iso) = 058bdb0e10e659b8970a65d2fc591077719258d540509e24f933e714226175314e394388bbaca613584fd5254b1c27c1bf3e348e47ab92c2e420991978a27183 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-disc1.iso.xz) = 1ea2b77200ed49074b0a4e3a7e91e3da36fb7b7ab1e941501b8c7c8797c18a6251d73ae9980781fe6614b2ca5e954638eefee11cfa44e5f9d1e0a8c882aa7579 SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-bootonly.iso) = 37f157344e14d69df98b4b945bc61e825813a77af8ee15e9d7c82f46f4bf56b0 SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-bootonly.iso.xz) = ef8cf86ea6b1e0ee87282b1aa4e58a4dcd3967dd41fba384d04a5776cd31855c SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-disc1.iso) = 033461d29a9ef13b4d7aff4d764d3733d228055154aad84d2288dd7763436df3 SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpc64le-20230818-77013f29d048-264841-disc1.iso.xz) = c69f72e0a506e4d189edaaee6c8e743638fb5ff721c00fb843491759177abe1e o 14.0-ALPHA2 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-bootonly.iso) = 39427c18814fc544da95ee9c3a8b72d6b29c435bdc7f015795a84757c3e9cb087ac3b4ed751c388934a103c4e4b0178aa013deafc57bc586b6fabc2194aad4d7 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-bootonly.iso.xz) = be9d357fb41973a4e369ce387450bfe7267d05e7ee2814dd31902060f185ef5ae8fc4f9279ad3e5b127ce250d2a073016a1e52bb93be4aea99e36e7ed3474f21 SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-disc1.iso) = 55fb63ce8202b154e5a92a4d5e8a60fccccc3baf6833f804910420f8bdfc8a7269b207de43763b78167186ce9e359bccf9eb750b25e67b1683a2f5daabc9e9ec SHA512 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-disc1.iso.xz) = 1757cddded61dfdc2d4961f184197cc555b81aca652c87f6113d0aabed8d09d744e8edca5300010544e77ed1e0586dff5fc2daf9e78f7d0f1d44c62d970d99cf SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-bootonly.iso) = beb1b05384c8252a472d4b08da97c97179d43b7d7b6018ef8fc41bb8e29172ef SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-bootonly.iso.xz) = dfc7396c28bae71d518d8a9a3d26210116c28d7d66a3d2f81d20ee3ed8667a03 SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-disc1.iso) = 8b2342126fcd5820804863384de9893bd44539666b2dc93fb8a9a9e253204125 SHA256 (FreeBSD-14.0-ALPHA2-powerpc-powerpcspe-20230818-77013f29d048-264841-disc1.iso.xz) = 60138d0f4dd1382567b3c7cedfb53686e624f3aee791244138fdcc069cad0659 o 14.0-ALPHA2 armv6 RPI-B: SHA512 (FreeBSD-14.0-ALPHA2-arm-armv6-RPI-B-20230818-77013f29d048-264841.img.xz) = 46cc383bdc70f7e8b251c528bc656497e354a1ef854035dd9919578a82fee0593ed354c9d28a059dcabee9fd8b48234fbe034a5a502d2eec412a96b2cf093dbf SHA256 (FreeBSD-14.0-ALPHA2-arm-armv6-RPI-B-20230818-77013f29d048-264841.img.xz) = a0aff47956ab1dca00e0e2cefe3584484b6dd92afed40d3b46bd768a0877472c o 14.0-ALPHA2 armv7 GENERICSD: SHA512 (FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img.xz) = 95e81b5723a6056e9a3c0551219f904cceebeb44681b7f8d0ed743e315a331011b577bde079e3dc629bf1d613dd565cd392d21785782bbbecefcf498cd375ec0 SHA256 (FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img.xz) = 90c7b36c9363f71171f90f1b497e9d9dc6f56559f7a72a87c325c2c4d209333b o 14.0-ALPHA2 aarch64 GENERIC: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-bootonly.iso) = 04cd332f1f2d78c65d9657a69c3756fe9ec06a0080b1045761628427e18da09c1581ad7b8c98db5bc92cc1b201d681a0d765c2a34a20713caf9cdc757ef89af1 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-bootonly.iso.xz) = 657fb4d00bed81dc9ad0bbc235056194551217b0186b064a275f78062c452d3a5e3ec31e11d1f91d97d28ca1982d1ecf252ffd9879488012715a573f0daa8af8 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-disc1.iso) = 17aa4baac883f2af9420929724356c69e3a0904f90387b71181b2f4ea9bcd1f8cb017eb7a33d0d70f41c2d519d354d97689b39522c74849acd2dcd066366884b SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-disc1.iso.xz) = 9b4a93d83068cd6c0b9f943db79b9514c8976bd3bac888290094062c6d565eba86e52791322b5db918b786d759c7886abbe0b35247aef4d84d36fb92d1c2de6d SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-memstick.img) = fb3948eebe632de32f4e409c2f3d5de2c568bfb7247e3c2054770b707a56b5ccceb333c230afa966619243a87196b7f983abc9932d736349092fd4ab04025bcf SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-memstick.img.xz) = f5ba08612a0ec2b72676cb592d085c919c3fe7e9a06edd53dbf3bfeaa7998280139d92ae49f652dd5f9842b4254897d5bcb5c77950468b2b3421301098ad30cb SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-mini-memstick.img) = 233822062a421d068a60f35e7c8deff8dc9a00511a47f2b9b30d698118c20ad736b5896d4d43aa20aaff6cee856359cce8536b32d5c75c34cd73a7ec2ac81cf8 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-mini-memstick.img.xz) = 45487694530129c9427d12fc6f60f7678ba85ec251951aa342b03b053588313b5519ec6fbadfbd9a0bd5c5dfd6e8bc66e19dac8e21f7e88fe5dc02ba4fb7edfc SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-bootonly.iso) = f3fca9dcd4b5810334c127ebd9b7958b34257f372339e2cab28b96f4e6e6d54a SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-bootonly.iso.xz) = 185437a35b922af82a50bb6ef3ab3340d0289f692dcb3bf57437ad8576f6268a SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-disc1.iso) = 91674bef26218b1f21b3fe765d16a9c74cf78b38c8b21666f6dd2a7157b49b1e SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-disc1.iso.xz) = 49b580645c7d97d4ce8a96fac649b14f8786da6e6748b0d38eaa212a5cf673e3 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-memstick.img) = c621141a32f2b1a018726f73a1e9c1bafc0c4b020e256fa01bad37e0b44b9ead SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-memstick.img.xz) = 2a97af188cc4f22c6c8482c4506d3dc9854d9e6a4ada5f08f978df4157bb8cb6 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-mini-memstick.img) = d557a4d78e12f7246b37a0280a8dfd1b8825b8c7699c9db87c158e134368d731 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841-mini-memstick.img.xz) = 48b409f032916c66dd7ce7da6ddef848b1f00611e64d36443f1295fc9918a788 o 14.0-ALPHA2 aarch64 RPI: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-RPI-20230818-77013f29d048-264841.img.xz) = dda0b83894cd514bc29693ea7a2e58d5485fc089924f0070b7374248d605807e63a7b00858ec4723d414417fcebb3496d2b394e08aa4d598f93d1139b93dc59b SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-RPI-20230818-77013f29d048-264841.img.xz) = 7851122bdf46e4e4b5b2c53c99ae604f9cb46bdb8894ca35f5486fb9e443edad o 14.0-ALPHA2 aarch64 PINE64: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-PINE64-20230818-77013f29d048-264841.img.xz) = 9ad36cf7cc7ba02b99994680599aeb153fabb488dabf41aa6e55279e4c9053c4dd610e1a290851d77c490e238ed9b19f2e259bddcbda09475b1e5d29e4840840 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-PINE64-20230818-77013f29d048-264841.img.xz) = cdd41b0e1992facfdfbd7e41d51a58caaf808aab290e3d7b2cdad8250d22c670 o 14.0-ALPHA2 aarch64 PINE64-LTS: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-PINE64-LTS-20230818-77013f29d048-264841.img.xz) = 5bef01310e03a4e44dbbef57ac133f9dbef7bf67e26d195ccd2cc75ad3a334aec0455d4ce47784705cbb1815172c15f97e34bcf5df3d69a38ff2219182578391 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-PINE64-LTS-20230818-77013f29d048-264841.img.xz) = 2d9d23c2f8067c9ce0089c27269608ed0293359237e1d19bae269c3f596d21d7 o 14.0-ALPHA2 aarch64 PINEBOOK: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-PINEBOOK-20230818-77013f29d048-264841.img.xz) = 3f27826293a5cb4b476c3609ac601acf2e400d4a2f8df600d92124f7a2df30567745173a947371942e683274aa0b6b44722baf8b047cd490b370163882354660 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-PINEBOOK-20230818-77013f29d048-264841.img.xz) = 4ac41b38fe4b078b5933a61bae0fb15999a77ac9dac2e597db8312264e4a5942 o 14.0-ALPHA2 aarch64 ROCK64: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCK64-20230818-77013f29d048-264841.img.xz) = c7fbd75c6059c88f4c3eea57f541e7fac05ae2d1810c53e742dca283c9a405fdbc2a795c95e53b6fd0d5ac455e5c6a1797c32eef96dc426ded8a5334d08d6840 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCK64-20230818-77013f29d048-264841.img.xz) = 478f30d2010916c5f0849c0ea8553f826c729980b86d2481a7fdbe957d18699b o 14.0-ALPHA2 aarch64 ROCKPRO64: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013f29d048-264841.img.xz) = ac74077ac0d84bb37454dc5b7817cbc91829c42bc2fc4647e944d0107a14908711eb09eba7dadf92e9d7d348faef2dc78e85eefbea9cad10e6230b03afc458c8 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013f29d048-264841.img.xz) = 129c485759e0f6764fa508ea26cbb1bff7fe85def9d1db55e487afa934d97c4e o 14.0-ALPHA2 riscv64 GENERIC: SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-bootonly.iso) = 9e98aa3c7e3ef457a28e28ae84da9f3a53616c2939402c7d64da24f6f40d3dfef49763e87e7e17ef6ce051e6ba72d65e92f6be7e259abade7d9e69bf1f94a310 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-bootonly.iso.xz) = 7253a76f796cac089017dd5dcc28a65a0819ba45e26a3246306090cedc578f140cc1d816b8ce97dafe614c0ebf98bb258fa81b2777fd8375af3e29731872bb82 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-disc1.iso) = a43a24275e9111069541d9730a3d495473cb049875602aa55d80d75a3fe9e5bb76590a3d6b2e0a8257cc560da7d844e1fd8eb6a323ac4345782922bf97142223 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-disc1.iso.xz) = eb5d013473b3ba76120526352daceb1241ff106325fccb3bd50398aa965335fe5ba05b3f0090c0403bf5e71b9b2a20f958d1d66c1de7800743b25be75c07db8c SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-memstick.img) = e7982113fb30cf3dd27f728d6a43da39d835ff4e214c240e26ee5f2e83ffdc332c740953a228a040ad83eaf2d4a7248b195cad35d91f15ba18d68a8a24a036b4 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-memstick.img.xz) = 361b32ca603715296495f1575838b19e85f133756cefb71a2e4c745c6724555a2bd909129864dfbc986bcab824e9d58b00f5c8e3d73eb7a6570f22d06274b148 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-mini-memstick.img) = 3c4bbaf22cfdd1f0d7ff930128016783e7f43f8c5dd306c9a4d88fff8af03a6fb4aa05330d04a26eb5f6e9de5d68df2d0f8cb164d00905fbc5390b85a82c590d SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-mini-memstick.img.xz) = e9583713510323daf7ba4e8622d70f00139e744fd3c285f408b100ae342b62a0f23857aa4fdce22b315409b7d49b00632a8e386137e0c5a553609ebd4fc99938 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-bootonly.iso) = c88c7300c19739e04c1a63939afd7cfc3f2934290416c0fa19394976239377a1 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-bootonly.iso.xz) = 0b441a28a206a930b451e0e33c9d5f63a1d420327b49168604b772c2fcb3570b SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-disc1.iso) = 82ef59e6a483aed9d138697b88e73a5003c246b8b58f68e18b6860152d4042df SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-disc1.iso.xz) = f3c54c14e24d90c75174f03567aad7c4402be0fa03e09c5b093a8de9ed9a4b80 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-memstick.img) = d7be770aec385c749ce4fe48637e28a3a1f3c46db39fb7a2418a3040849fb78c SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-memstick.img.xz) = cd45a1e0bf984e3a43ffa72968d314ecac2968184562ab5a962a74ec991de689 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-mini-memstick.img) = 179beee77cea303820c0994987780fcf00d53276be89360a54504ab611db865c SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841-mini-memstick.img.xz) = b47827f17058827036f62d2d73f15940f762932fd5f9d20bbe35032f0bc6a33b o 14.0-ALPHA2 riscv64 GENERICSD: SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-GENERICSD-20230818-77013f29d048-264841.img.xz) = aeab2f40a86049ca0a39ca0ce15b9193b272e16ee66a91f959011e65adda5069c4f4020ede5d32576e40e23271cb3bbb1b52f3670226a16a9ca1d715e4b3a61a SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-GENERICSD-20230818-77013f29d048-264841.img.xz) = 1ed1da256eb3c8bf514568a0fca1adf700f70ed055df7a9cf9af9a495dc71d7f == VM IMAGE CHECKSUMS == o 14.0-ALPHA2 amd64: SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.qcow2.xz) = 1cb5b4d75b170076a443761fc6efdb9c5d88781501557aba78a23a32c7c56d508e4e8723adccf52ffc384e8230aa997f74ffde090c66b7737c7966a6b785c1d5 SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.raw.xz) = 06010bf22be9ecdc4654cb4c040f7ae2aba709f8ff8aae485850ba53e7dd8370e25b160551bff318be047c16dda257e0de374b5b3ba300eb73f75cab97b3048a SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.vhd.xz) = 9d63076b5135d7c517885271fb0f495a37897fd93d3d9ed761afda68fbb015a5aae31b3b5831b72388f1d2ee37f573eacfd5872d8c52e248bdef4bad61df048b SHA512 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.vmdk.xz) = ba4715d919e258b1e94baa5613eac176c2bf261db2ad856e5727de35d0f57614cc02efc5bc584aaccbc34804ab2a1f19e705553c1c9a1c11d73dd52858d4c3ab SHA512 (FreeBSD-14.0-ALPHA2-amd64-ufs.qcow2.xz) = 1cb5b4d75b170076a443761fc6efdb9c5d88781501557aba78a23a32c7c56d508e4e8723adccf52ffc384e8230aa997f74ffde090c66b7737c7966a6b785c1d5 SHA512 (FreeBSD-14.0-ALPHA2-amd64-ufs.raw.xz) = 06010bf22be9ecdc4654cb4c040f7ae2aba709f8ff8aae485850ba53e7dd8370e25b160551bff318be047c16dda257e0de374b5b3ba300eb73f75cab97b3048a SHA512 (FreeBSD-14.0-ALPHA2-amd64-ufs.vhd.xz) = 9d63076b5135d7c517885271fb0f495a37897fd93d3d9ed761afda68fbb015a5aae31b3b5831b72388f1d2ee37f573eacfd5872d8c52e248bdef4bad61df048b SHA512 (FreeBSD-14.0-ALPHA2-amd64-ufs.vmdk.xz) = ba4715d919e258b1e94baa5613eac176c2bf261db2ad856e5727de35d0f57614cc02efc5bc584aaccbc34804ab2a1f19e705553c1c9a1c11d73dd52858d4c3ab SHA512 (FreeBSD-14.0-ALPHA2-amd64-zfs.qcow2.xz) = 8e4038881e4d05455bbdca0c2cba9e93fb596bd830c655db447bdb713bb36dbf7dc7b7e24609b8e5977059c1dda50ce84eb6e60efabf30f5d05c61118b4fbe85 SHA512 (FreeBSD-14.0-ALPHA2-amd64-zfs.raw.xz) = 616606984a5f4f57a73ea38f0746d6764db81734bb5cf17d36476e9d8067064e46c2e365f927d71b18be0f369087696d047b228965c4510a2b92e96a12f1dfc6 SHA512 (FreeBSD-14.0-ALPHA2-amd64-zfs.vhd.xz) = 752716f8618c8e48398db1b6755011fffb1c36d18314fea51556b005f0bcd6f8e484f9ded60686fe0223fc8825f27b79ecbecf2257faa465e3a7e2b68556390e SHA512 (FreeBSD-14.0-ALPHA2-amd64-zfs.vmdk.xz) = e04c9435de1063ff5f950070893cde6ab98421221c9825225ffbea8ba4b1e30a8729bd9306ab8e4007805f2ebc421b1decc3b9d7d6b752fdc29843c0f8a8a339 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.qcow2.xz) = bd36bb410dd34d9fa70c04380a8b0c5e6cf2cc9f5aaf0b82b76ed74f54ffb199 SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.raw.xz) = 0b64393ebd2f1916b1fdd3a343d833e731b8b1579a82b486c0e4aa9bf496ac2c SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.vhd.xz) = 8d46e8d76af9649f15afab7f69b712331ec8238a8aa6ca825e5df6e551d829ca SHA256 (FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841.vmdk.xz) = 2bebd0cbf15d4414069babe0c3a360c62198d10682ab69ae06dd53ac9ff5f247 SHA256 (FreeBSD-14.0-ALPHA2-amd64-ufs.qcow2.xz) = bd36bb410dd34d9fa70c04380a8b0c5e6cf2cc9f5aaf0b82b76ed74f54ffb199 SHA256 (FreeBSD-14.0-ALPHA2-amd64-ufs.raw.xz) = 0b64393ebd2f1916b1fdd3a343d833e731b8b1579a82b486c0e4aa9bf496ac2c SHA256 (FreeBSD-14.0-ALPHA2-amd64-ufs.vhd.xz) = 8d46e8d76af9649f15afab7f69b712331ec8238a8aa6ca825e5df6e551d829ca SHA256 (FreeBSD-14.0-ALPHA2-amd64-ufs.vmdk.xz) = 2bebd0cbf15d4414069babe0c3a360c62198d10682ab69ae06dd53ac9ff5f247 SHA256 (FreeBSD-14.0-ALPHA2-amd64-zfs.qcow2.xz) = 02a00c67b588cadc19457d90e3933234f1fa97bf73994da0470b829a63d3a5ae SHA256 (FreeBSD-14.0-ALPHA2-amd64-zfs.raw.xz) = 836f40f7b5aad52a3c0b839c6a879bad1a8745f46c152b880ba4daf87416342a SHA256 (FreeBSD-14.0-ALPHA2-amd64-zfs.vhd.xz) = 050a053ac20084c9cd1c3b16d666546816ac5db53214f56076c4bab78d7ebd55 SHA256 (FreeBSD-14.0-ALPHA2-amd64-zfs.vmdk.xz) = bf84867bcf0755b7a149d11e531348505a8b6ffaf973bcc7ce9378875b59eb22 o 14.0-ALPHA2 i386: SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.qcow2.xz) = 69ae7347622a9fff7993838f59e911c5d2103ef4e97e9a2aa630a1d808ddb2c39e2a1008c1948ebc613d66c7e70e2e4e4320a3732b1eb8ed97cee52243dbc2dc SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.raw.xz) = 55b275e644ef06e2373dbbc5833ce470693ae4318dc6a2d8bab6a29188eaf9b6b3d7678e78eadb617b70ba54f294d04cfc8c4ad7aa6466f76edf3257587f6856 SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.vhd.xz) = a24ae4711668511b0ab8c8d7618f044926a819d8e862f5772787c78febd09050e63543c55ef2bf7c3353a358653edae81d17d21398dc3cc719f168f2a3d6a25b SHA512 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.vmdk.xz) = fb70db6fbd08fed3c04f664f069add93b09537298f298eb8f40c84ea39edefa2ec28d91906969d92dd3c79fcfca48d5637aafb65baf66787b07ee982b32a9404 SHA512 (FreeBSD-14.0-ALPHA2-i386-ufs.qcow2.xz) = 69ae7347622a9fff7993838f59e911c5d2103ef4e97e9a2aa630a1d808ddb2c39e2a1008c1948ebc613d66c7e70e2e4e4320a3732b1eb8ed97cee52243dbc2dc SHA512 (FreeBSD-14.0-ALPHA2-i386-ufs.raw.xz) = 55b275e644ef06e2373dbbc5833ce470693ae4318dc6a2d8bab6a29188eaf9b6b3d7678e78eadb617b70ba54f294d04cfc8c4ad7aa6466f76edf3257587f6856 SHA512 (FreeBSD-14.0-ALPHA2-i386-ufs.vhd.xz) = a24ae4711668511b0ab8c8d7618f044926a819d8e862f5772787c78febd09050e63543c55ef2bf7c3353a358653edae81d17d21398dc3cc719f168f2a3d6a25b SHA512 (FreeBSD-14.0-ALPHA2-i386-ufs.vmdk.xz) = fb70db6fbd08fed3c04f664f069add93b09537298f298eb8f40c84ea39edefa2ec28d91906969d92dd3c79fcfca48d5637aafb65baf66787b07ee982b32a9404 SHA512 (FreeBSD-14.0-ALPHA2-i386-zfs.qcow2.xz) = 900e2e2a78b4c731ef0907b18a5666d5bec612dbe5612d8417b22ba701b6cf7f702121785b6eac3be09ef31259fbfd9512ea21e80561f00a103125cf6e33346d SHA512 (FreeBSD-14.0-ALPHA2-i386-zfs.raw.xz) = 1ff9a29460c18ad7df06df6a2120162df78516c9c016c39e4453c7d939fd58b0407c6469a043db0e14933a65b7aba7ff0df81bcdd8fe6749f68e9144001f99e8 SHA512 (FreeBSD-14.0-ALPHA2-i386-zfs.vhd.xz) = 471ded3468035c153582219aef368f0f1d706a6fa9c70804e18abe9e8ac38d1d77ad2406ff816e76225f396ce4bcef1bcbb3c4d9d7d67d1236558774745ebdfe SHA512 (FreeBSD-14.0-ALPHA2-i386-zfs.vmdk.xz) = 6808e52d245bdc45da9394b1686caef2fac7522c1a6f8daf50f99855e9522a6db7bdf4ad5fc1a092cc8865bf8df986e902040b43cc59f411f3533e38ef55d6e2 SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.qcow2.xz) = 2388bc91b59a97c075a7bcab8b8d078e20b7c5d33caf068b2958182c907cc24b SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.raw.xz) = b89dd4435043e4043d408f3735a8afca21d62b98e7218373f982d5bfea1ad396 SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.vhd.xz) = a6894a561afc494422e8648bf0f89e8fb22f024922879777f43dee4ec02dc8e5 SHA256 (FreeBSD-14.0-ALPHA2-i386-20230818-77013f29d048-264841.vmdk.xz) = 47996f44e10d390b875a0ab2ac61a20979412b86fcba9b81a05f93a00a16c01a SHA256 (FreeBSD-14.0-ALPHA2-i386-ufs.qcow2.xz) = 2388bc91b59a97c075a7bcab8b8d078e20b7c5d33caf068b2958182c907cc24b SHA256 (FreeBSD-14.0-ALPHA2-i386-ufs.raw.xz) = b89dd4435043e4043d408f3735a8afca21d62b98e7218373f982d5bfea1ad396 SHA256 (FreeBSD-14.0-ALPHA2-i386-ufs.vhd.xz) = a6894a561afc494422e8648bf0f89e8fb22f024922879777f43dee4ec02dc8e5 SHA256 (FreeBSD-14.0-ALPHA2-i386-ufs.vmdk.xz) = 47996f44e10d390b875a0ab2ac61a20979412b86fcba9b81a05f93a00a16c01a SHA256 (FreeBSD-14.0-ALPHA2-i386-zfs.qcow2.xz) = 0218bd04b58bd16cc01ba2a168692bfc8298551f2ee72dc78b75df5d7fbc2c0f SHA256 (FreeBSD-14.0-ALPHA2-i386-zfs.raw.xz) = b449ad255b8ed8448fb3ae855ded0cae8df332ec318dc79d9b67cca3a47407b3 SHA256 (FreeBSD-14.0-ALPHA2-i386-zfs.vhd.xz) = 4a74ce6fa656c446e09e96d7cae8c0372760e9c00bdb98ef19a115d395958c11 SHA256 (FreeBSD-14.0-ALPHA2-i386-zfs.vmdk.xz) = d55612217b4a7b9a96fe9d2762a904dec4c414832bf9caf9b51d9cdcb5610bf1 o 14.0-ALPHA2 aarch64: SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.qcow2.xz) = 103eb6042d98077fa5208c4280c33dbcbfd0ec9b579abdc17eb511397ff790c90511a2ea4246285194e9e1320a0f155574293618ed69f8f1d03fde4888e53d3b SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.raw.xz) = e3d324efa0a75b6e9362b072eba1d32d138571132f1f80197b4c4c1532c3db4092022eed1f086dcd4c3dd383cd77c898cf47221e071494d56eda38d299e63398 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.vhd.xz) = f3dcda3cd46f1c9cf91766390938ed921ccf78828e9f297ada55410d3f8d103546942f303074b9ffe6dec09c3232bc288ffbe7cac66bae786d7986490d49c204 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.vmdk.xz) = ce0bbb0a238077f098d1aeea596616c456ff1df713cd28a1db50b52a060c2de96103603e8c09db87aa71f2119e3d068f179709e6d4f1561d42d5fe8ad04d83ac SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.qcow2.xz) = 103eb6042d98077fa5208c4280c33dbcbfd0ec9b579abdc17eb511397ff790c90511a2ea4246285194e9e1320a0f155574293618ed69f8f1d03fde4888e53d3b SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.raw.xz) = e3d324efa0a75b6e9362b072eba1d32d138571132f1f80197b4c4c1532c3db4092022eed1f086dcd4c3dd383cd77c898cf47221e071494d56eda38d299e63398 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.vhd.xz) = f3dcda3cd46f1c9cf91766390938ed921ccf78828e9f297ada55410d3f8d103546942f303074b9ffe6dec09c3232bc288ffbe7cac66bae786d7986490d49c204 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.vmdk.xz) = ce0bbb0a238077f098d1aeea596616c456ff1df713cd28a1db50b52a060c2de96103603e8c09db87aa71f2119e3d068f179709e6d4f1561d42d5fe8ad04d83ac SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.qcow2.xz) = 8147ef5fc71783d846ffbf16f6c281838de0de20de989350519a94ab783fd99503029dea5cef980b629aa448f3ad855f9beeb61aa0191f67a44158a164f70481 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.raw.xz) = 0fc7b78421e91ec7ccf3f7ddbf2332764eca63a1659cb5a16bb9a373ff664bb1aa411bf05121b13c3105e9a71f8139e146b744eaab8e4be812253c096aaad298 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.vhd.xz) = a97bebbe5a941186ed981778a41c695880d5815f814f47017e3902285e216131333fdced3982216754af5ac5ffd369894bf9c34834045646efcf702e33823364 SHA512 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.vmdk.xz) = 252722e85090919bf2180968957806da13b21eefc008c80cb28f502ea543dda9bfb5b5efe60b5f9be6f7f245531d02d53bf1ce164209161b3994f08c7ed80e37 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.qcow2.xz) = 7e1eb4aa7cac14aee4ccc6efa3fba66269da7b317f59131b81c8a793e29c8d0a SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.raw.xz) = 90ea87e22e6b0e2cdf5fdbd0aa7cfd250120fe84c046e21b368148b2ad20fa26 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.vhd.xz) = 02a91758594aba011b1dde896a60fd7785576c079120faaa25d27c6857212d9f SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-264841.vmdk.xz) = 85ee99bb8650f47d43379d286631172c66647c58d67bb7903507e3c08dfbc3e7 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.qcow2.xz) = 7e1eb4aa7cac14aee4ccc6efa3fba66269da7b317f59131b81c8a793e29c8d0a SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.raw.xz) = 90ea87e22e6b0e2cdf5fdbd0aa7cfd250120fe84c046e21b368148b2ad20fa26 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.vhd.xz) = 02a91758594aba011b1dde896a60fd7785576c079120faaa25d27c6857212d9f SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-ufs.vmdk.xz) = 85ee99bb8650f47d43379d286631172c66647c58d67bb7903507e3c08dfbc3e7 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.qcow2.xz) = 01afa427609e884b8819cadd8a08d21a264db19076cfbf0d04e6739a8ce3829d SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.raw.xz) = 269ba1f71dbc1447abca20adccedb2d5eb76c570caded7f67c37d2c6042026fb SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.vhd.xz) = c7c935c8c65239672673409f98983a23abf752046ba81c3291c464147d39fbb6 SHA256 (FreeBSD-14.0-ALPHA2-arm64-aarch64-zfs.vmdk.xz) = 14c28b434be7323d98c3e7c68caa6da368c2a522e3261b7ca9440ea29b23ea62 o 14.0-ALPHA2 riscv64: SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.qcow2.xz) = 1d910e517780e3cbd750655b40cc56a9c70d496422d7267bb5e3a2bd30247f8462c6d773af5bc2a0f167f9ad8ff363d4498084a8b6081deb3ddead0ef53d570a SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.raw.xz) = 0ac8eea970c4a8db156b39fefc65327ee00f5fc2d65aaa10cb118b4ee4f02d6655814f37c10cc9822d7201ba0e241b409c0bae149bb0156ccdb6b36fdf6a9a18 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.vhd.xz) = c2a72fb202215a70d37a1f9614ea4a1728a8033f0947d4e4d0dccdc97518a90ab8ce06e4bd864c5617b74d7fb6373c1ae6a6ffb4f20bdf48e1dd376d981574c5 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.vmdk.xz) = de32fc2ad77d29b9686bb2a1c3c77aec0b941457853f03fffe6e32f6204ad562c35814ab574cfcb014cc6b4835857c1dd4372614b345cbf0d1d200467745947f SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.qcow2.xz) = 1d910e517780e3cbd750655b40cc56a9c70d496422d7267bb5e3a2bd30247f8462c6d773af5bc2a0f167f9ad8ff363d4498084a8b6081deb3ddead0ef53d570a SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.raw.xz) = 0ac8eea970c4a8db156b39fefc65327ee00f5fc2d65aaa10cb118b4ee4f02d6655814f37c10cc9822d7201ba0e241b409c0bae149bb0156ccdb6b36fdf6a9a18 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.vhd.xz) = c2a72fb202215a70d37a1f9614ea4a1728a8033f0947d4e4d0dccdc97518a90ab8ce06e4bd864c5617b74d7fb6373c1ae6a6ffb4f20bdf48e1dd376d981574c5 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.vmdk.xz) = de32fc2ad77d29b9686bb2a1c3c77aec0b941457853f03fffe6e32f6204ad562c35814ab574cfcb014cc6b4835857c1dd4372614b345cbf0d1d200467745947f SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.qcow2.xz) = b13dd827f92301c2e7c211e3ab820b3eb36fd07ddb853a7e4f6750e22360b770bd38e665476241844f1e9bf0483438e7bd92091deaa77bd7c73acc1d99f2f695 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.raw.xz) = 270061e08e9c4be3c910fd829e2b50360e4e3a347d6d26fee1cb8cdbd17646b82f194357d5fea7276ee9b364b726990828f4c89e33e9d9add7cee2f5dc75faf4 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.vhd.xz) = 89774371421d27b06429f9b95bca17097c1851a14dc2ccd78e592157d95f35b9ada3ca24a8bfca746653c149acb32ff9a4fdcaf0c01ac785e427a57ad6b26ba6 SHA512 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.vmdk.xz) = 09aeb4bb6b932baa49c1751f6036d9ede019b184e168214cb8ad2a2e0a16cfdf708deb8d2e41483ab3d5b6ad98b0e31ad01fc17a69abd02fdb6ac445a5d47c25 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.qcow2.xz) = f3e307136e471a5a58e4fc341259efa3bb4b105ec3aeb7669f9b34593cbf5417 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.raw.xz) = ce52a70708c1476185d45b4e280754d863660ae80b452d7afe4bd35b896aa864 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.vhd.xz) = 7635f35bd6eb8df3bc9d72ea1753a35ead36b670fbaac66508f375046f69229a SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-264841.vmdk.xz) = 95f47858ceb8e99b6d9c1f644f69af39919961db14d73bb4781965e01744bd6e SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.qcow2.xz) = f3e307136e471a5a58e4fc341259efa3bb4b105ec3aeb7669f9b34593cbf5417 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.raw.xz) = ce52a70708c1476185d45b4e280754d863660ae80b452d7afe4bd35b896aa864 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.vhd.xz) = 7635f35bd6eb8df3bc9d72ea1753a35ead36b670fbaac66508f375046f69229a SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-ufs.vmdk.xz) = 95f47858ceb8e99b6d9c1f644f69af39919961db14d73bb4781965e01744bd6e SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.qcow2.xz) = f5c2e506b0cc48061da2818eaf72a2d8a0ef52025a8504fc236b7a1d94977b90 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.raw.xz) = 72e182aaa4f9e016e671eae71c2fd7792e67127c825da6ef5dcb7544c1cc3581 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.vhd.xz) = 4a96b090e02f644d1804fdf98f063786f3981222ba60350d9b87bc698e0d91d9 SHA256 (FreeBSD-14.0-ALPHA2-riscv-riscv64-zfs.vmdk.xz) = 04be6f7c9c2eae766160ee7e5129a320a82756c20a977c504d236adcb389e30e o 14.0-ALPHA2 amd64 BASIC-CI: SHA512 (FreeBSD-14.0-ALPHA2-amd64-BASIC-CI-20230818-77013f29d048-264841.raw.xz) = 1bf1295befc237c0796cc1ef51e001dde16ec6e801efd033178bba8a60c140b462ac944b0c26a7f1b7c42ad61801a5c7592bb7ad9db2276d828696882150b595 SHA256 (FreeBSD-14.0-ALPHA2-amd64-BASIC-CI-20230818-77013f29d048-264841.raw.xz) = 39b56035b8d1a1883663894ae08d71a42cc530d58b8cb0916407d9ab69581207 Regards, Glen Please consider donating to help support my FreeBSD work: https://www.gofundme.com/f/gjbbsd https://paypal.me/gjbbsd Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmTgIMYACgkQAxRYpUeP 4pNvbRAAnkRIaaIebigc484y4kqomId02LP/phczeFQngy48Rrfgrf5Hb152DG0I VBTeqf0IFgUhI9O9J/oov8mkvsDGhhYRSeZyiHAg2hJ/ypP9TMdENYMYuVLGK2sK FhvEpDWc3OW8Ep7FIQQcUpWX/GG2DrCKKbX+Cx78kfCFpM5P+6eXecg6WiOVLf4H JQzkDXraKQnzOWYcpc8C/wsI14Rp7qNYA9sOgGnokITjuiKyqWzRWUJ9ZRAQFPeL 7c03XweJrhDIniICjy7EY36//ntBJeCx3LUNPYrkiSmZ7BRBUl0JievENDqOV2DE a8VD942bCvE1acHjbcGcEh8ASaVxbMFnTDa33XOMbKRCZiJRXOyvo26xg0Dk+zN5 +PmGM72NXLvG/GSvtsN6o5o0C3nsp5Awp4TqvPYF6vI7gzVh4AUyTcmNlHJo/w83 d7oeG0N/EoA4BVAH7v0r7XdD90ocIAadrjRxDZRy5gScG56tTmgS5g5xM6RnNyt0 AUQyItksT5/Z+b3TRkpvXIyPQG5QI6OfgZ0C3YX74XGWMFFN2Dm+j+/v5CGOaXeq pQFUf1m+naNELLrquSWR6bDL8FBwXbxJEpCmn+hK/WnZAu05ejPaVv8vwz8BBWqf 2f0+6xYWaVd6qOyzao7B5S5j4iCmwNlquTXMMSRgF5PyirX69+A= =vr3i -----END PGP SIGNATURE----- From nobody Sat Aug 19 02:09:10 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSMcm6l4Bz4g4nf for ; Sat, 19 Aug 2023 02:09:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSMcl21B0z3RHk for ; Sat, 19 Aug 2023 02:09:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=Qv4PUjb7; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692410964; bh=qg22xNyQepmNaljF8gUhYKG+OpZ9ELPN5yKCOvJ2QSA=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=Qv4PUjb7C0zMZQ+IbvS5uCD6ScaOY5TePQEerKm7qFFH7tHKhBGAgHtshHJkizm2XvlUcFXNBBmg20HGBuOuHP54wICClPFG+Behzywy0iKwdG3BWZBwnUsYBI7v2PDF6Wyft+GG+06f5smGCLKd1xyLfcsILTEpZh8bDOkigIlkxH2dY3crm+XTPrWW0/6r7zXwxwq3wq3vAJWau8R16OAmW3a6v+JhFI5r6R9uW4kBq3WfV1nvn+OSUec8uYMPuHu7xkrBxnZuFW89kvLsGYpQqZDvZdb0jSWfEoYO+30ycGF7uvaq9cORuQ95cou5wcEisgzjbbl8QsQ7VgikTg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692410964; bh=QNT6lAoANQgG2sDHTT3F5VijVNjiG90wfAxrmGXZX4C=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=eLil2h9R61tZ7UYFSlizeNEPRvtvJ1X3Y6JBXzbyrU2AYIhnuInDLZp2vNWOmTcQlkKg/yrMaRXB8I2tV0DuQe4ZDBVDomJHLlEb3YR2vD5Ar08qjLF5UwGqwAohyKtuwbIsMW/I6BxJuGmCXWWxZ289WAtjz3zOchqxlqVSQR4QWO9Q032P2wLmYk4uoAEliepKO3WrSiam9kNgFCBJQcZXgtvx4aHloMkxgmlDs87eHjLUGpDjrYEOGCbcf/te2qdyLfk8yi1nnrba69UZP+uboyQ3e269HIycbFrg95JvBACD5b848lPw84rjFwRSXCOPczWGIw+Cln48KKI7lg== X-YMail-OSG: iNdv0ooVM1lQmFvd1fKH4uHTm5I6SsOpFfxM4ZpVxCdqaFLyY8zxNt5CD6dBuev xR1pNC8RMh9bI5SYP8sJ1_o3LbRU28gHK4_vs8aKag4cm.q56ps6DuRpt1KqKG.xW53rE9nyLrhO zbDQyJJGNWWDaXxPuNCXrW7nNDSR1u0LgL6yY5uCosGUyM8JLU0mSMrRbUyuRnyHaf2o99afo0CQ tmgpn_oogEpH2UZXARPRNPZiIy4SiUHIQOhtRzPaWewxZWELTTi.URLFDb6wJqWGQPOxn1UHyJW4 2DTnuuyfV0eS6WFenxuRMxUFhPpLUq8ZiWDIHUyYGd9o9q6IpWsfiYhlpN_eTZlwsWjecE0AYNA5 RMgRYp2Z7KnA66pCaS.smsjWi1AGU5CRQc.alv46Xy5kRPOi87IO1rYSWSC4IAoDz0g8Nq0UtGNu la0_eYSyE9l330jqVOrHG8M.Srr_yIGdrJ0I3VCrhaGLP8.ll5TyP3MhKHX0F8UZ4qFZQgqKycKj KTRqV3VML3zf3swvYmJY4vjdXVeIYx7ZuzZHblhiTTGjtFOhk.GQV5eawJw9UMUeZQdGUOEBkzHs e.HkCqbzKueEvttY8IIoq2LGSJuW_XT.Nb2TsbqkUdFZEnEgq9Hqr20unwIYzD7ijjAhlVwXLXK2 XCtuu1SksQYcOBoClhqIJsF2Gn8C0m2KqIV6zWf5M17LFa4hJs.p8vZgXy9XT_VWgMgXmoDh7Akp mRHpZFJU_r9ukN7LFITZywOobSXjIZOoJJTwnnwjX34B5PrjmrMC6OoqPGR7ri7d.dN5oDu3A2hP a213VUaErM6fsl2M0FM2SYFgJgtdAenKCQ4.P9MeCrfNBmi4KHVKL.pVCXVOf6Ti6SmG18DAeOqV xM.j2w6J_bRBMyGNYf5RqB_oGznAe_XuAfd7NX2RhC7izIvbUUcZjJzNe0_BoDRNBTdI8_hDvaze dVtAwvDJ_jzdsOZZh7paK765Gl36iyq_amiNDZEgNszdtBxzAiPl7dCTikfadf57emZ0o9clLkVF I.JQCNV3E32Xl4MayRvvm56P4ot11kpfc.V2nd.szH6KQHX22dQ_lJW0p0fAFNe.rMOfQkPVZDcW QeUgiJOu9iEsZkxSPZerccc0WAKFIC_xbGSaVd.3JKTv0KSQQVX7Ii2HDwNkrPlGQZe60RBD9.kd jYj7TA3rHp_7rby7xlmNeZwjzHD19RQcTU4PJxlPNxPsONYCBDpikUfpfA4nghTGl57XhV_eja8i pRe_8yboqgfX1xbZg9rAgmhDKQvl1Lp5YcriVhf9gmMuNboS9pnkka5lnUqrG_2DaFsCLveaJRTH 94dF.08A5hH1YB0emsDnFsRA1n.yeLkOhIZ1AQluQ0ayaOcZ8AdHIgKVordwJy4xBdmRQvFDZKUq vr5GnoSuxHXVV4ZeVzB4Kv9trv9lMXOhoIbuCCSNRqqMsmNWdyc8HHOybcZCNOfsCRwD_6pf4zGx 3vT5VpEZ_KV39hdexKVAh9WUeTJkDisCVhembm8cv.UUjmx.zZ74y6gCMXNYxODY391GqNEnSj96 GqoeAIn6foS3CTBm.pRnVgkrklObSPzGU2LwIPbGRz0cs_O9T0Ln_XDlezBZdrNkkqDuS7QouwWu xQ9W9j8cPcL2LpWPFdHEnsCavoLlDAoWEWwHk9r7kYfWU_d22fG9zukPz.9bbw64UuyuGMu.d7ZT 90OdHBmST1G5Cczs3aMWmeDIhLfgUIolLTy_JpE947CQ57m69t3e24mh0KteslKKrEVsoeCGHlng PQdGQMlYpeL5LtYVajixAHpQCYO4Ol8TsmHImG9B4IF2oOfM0ZEu4zgXwR9VhfIFotg7JK8AEzm_ tfPyLghUJhIBAQANNqzreXEz7gGAyhgAeoEsYhhxHuGV6hDMshKROXHN3LnZ5eFGcQaP0qqW5QwP Q26NdAXwRxkcqBH1g7esN6VGHU0z7Osc_37BxouKK3mdD5PZlPkcgFeZvWAkbW.BvA2SLRRv_PKN FjFjdW2l4GPWOd5JPDxIGtlb5bjOJZjZtY1NeljEbvkORtMas_JnEMohB7TfPKG9ACR.ZOBofM9W Wx0lDtga29qDLYg0HtCHElLBxDkaHaoOYaHfq2_4m6iFDj9x.GqwjAcMtbj.8AxDU04eXUMjTEMw iTSWGRJqb2YynE8BQTZT7C7I7zcz4CVUaCgyGsbMYkMnRuUSjfYIziH9YaG49c9Z2FTG8oEdb44y ncrwpV44icgshYQvIbxPqLuI0WN6t8sN.LTpTeFS0U3P0AqGpj6xAex8MOlcH.W8nFQHLtg5oUaj j X-Sonic-MF: X-Sonic-ID: 667af094-968a-4eac-9a05-e67f2bac61f2 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Aug 2023 02:09:24 +0000 Received: by hermes--production-ne1-7b767b77cc-27nt8 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3bd7c4e696cc0fc53140f24bb44348a4; Sat, 19 Aug 2023 02:09:22 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Possible regression in main causing poor performance Message-Id: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> Date: Fri, 18 Aug 2023 19:09:10 -0700 To: Glen Barber , Current FreeBSD X-Mailer: Apple Mail (2.3731.700.6) References: <8A267B4A-1B7B-4D61-AA57-8E3156470617.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.31:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.31:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSMcl21B0z3RHk Glen Barber wrote on Date: Sat, 19 Aug 2023 00:10:59 UTC : > I am somewhat inclined to look in the direction of ZFS here, as two > things changed: >=20 > 1) the build machine in question was recently (as in a week and a half > ago) upgraded to the tip of main in order to ease the transition from > this machine from building 14.x to building 15.x; > 2) there is the recent addition of building ZFS-backed virtual machine > and cloud images. >=20 > . . . > The first machine runs: > # uname -a > FreeBSD releng1.nyi.freebsd.org 14.0-CURRENT FreeBSD 14.0-CURRENT \ > amd64 1400093 #5 main-n264224-c84617e87a70: Wed Jul 19 19:10:38 UTC = 2023 I'm confused: "the build machine in question was recently (as in a week and a half ago) upgraded to the tip of main in order to ease the transition from this machine from building 14.x to building 15.x"? But the above kernel is from mid July? (-aKU was not used to also get some clue about world from the pair of 140009? that would show.) > Last week's snapshot builds were completed in a reasonable amount of > time: >=20 > root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G > 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/logs > 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/chroots > 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/release > 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/ports > 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/doc > 20230811-00:03:13 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release > [...] > 20230811-15:11:13 INFO: Staging for ftp: 14-i386-GENERIC-snap > 20230811-16:27:28 INFO: Staging for ftp: 14-amd64-GENERIC-snap > 20230811-16:33:43 INFO: Staging for ftp: 14-aarch64-GENERIC-snap >=20 > Overall, 17 hours, including the time to upload EC2, Vagrant, and GCE. >=20 > With no changes to the system, no stale ZFS datasets laying around = from > last week (everything is a pristine environment, etc.), this week's > builds are taking forever: My confusion may extend to this "no changes" status vs. the uname output identifying the kernel is from mid July. > root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G > 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/logs > 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/chroots > 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/release > 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/ports > 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/doc > 20230818-00:15:46 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release > [...] > 20230818-18:46:22 INFO: Staging for ftp: 14-aarch64-ROCKPRO64-snap > 20230818-20:41:02 INFO: Staging for ftp: 14-riscv64-GENERIC-snap > 20230818-22:54:49 INFO: Staging for ftp: 14-amd64-GENERIC-snap >=20 > Note, it is just about 4 minutes past 00:00 UTC as of this writing, so > we are about to cross well over the 24-hour mark, and cloud provider > images have not yet even started. >=20 > . . . In: = https://lists.freebsd.org/archives/freebsd-current/2023-August/004314.html= ("HEADS UP: $FreeBSD$ Removed from main", Wed, 16 Aug 2023) Warner wrote: QUOTE . . . , but there's no incremental building with this change, . . . Also: expect long build times, git fetch times, = etc after this. END QUOTE Might this be contributing? How long did those two "Checking out . . ." take? Similar time frames? =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Aug 19 03:20:23 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSPBh61Bbz4q49T for ; Sat, 19 Aug 2023 03:20:28 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSPBh5Mt5z3Xys; Sat, 19 Aug 2023 03:20:28 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692415228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kqHhSiq6pqUxB/RvMCpT/+9kOpPv9RK1rM/lSBBhjIM=; b=hUdtHyUzPpbm6HhsarCMbWMCL1sYcjkTgBoVRSiXonugJaC9x0KzfZFQziF+NA8Xm3z37P CnoxiUZrtaTWEaePn2UcU5ECZXPmcJ8db8VnqZNhICWydWPB9qAIWCe3fs3xlyj1ypFzic oNev1ML+IPtgpI7gERJvsBWurM6EpwFtKywvG7Guejx8w2vh133EjM85FdWVX+Lj/WxUP+ RSToBt61zT1GoLW+KgB1/O+eP/9mttW/hUBcMxhGx42WQMnNmw4PmHD2bkO6/rIsCLNFjp UP0RBuC61mEzIphybZ/KCQjbHLY2Z3X77PjuSSpjDXCJv1jllnyz9bQAoBNHgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692415228; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kqHhSiq6pqUxB/RvMCpT/+9kOpPv9RK1rM/lSBBhjIM=; b=cIL7vwLQ0BjbXvF/1BLvlvLXPHw4Re7AWUlL+O8izSQQeTOQayAXLz6L37CCbDC6v5knSg S/LsArtTUnL8e4ennr1hMH0YTv9pPmr0RIWl3wXGmQ0E4AlfpV/xWO/OI4awSH7++4cepU z5tlH+SKtY79/P0HYmsWwz451Ov0FEIq8lhvKG/szKfLwFujl4r2jUamBMbg9fJ/CVno2F OPswN0ybplDhgwX2OBNswP1HKShTro0HhrloEySuF/LAerKtU50D/iJZYWS1hwAlbnegl2 T5JVg0uhPwu7xIoLoQQDECbtZk3r5Jv4r4Ycuo2+IcJpwbX5OSjbPL2+08zghg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692415228; a=rsa-sha256; cv=none; b=r9fs+Z8aQZzhzfuN4VTyNGGZfOyCH5feYocFpopnlMXE3yRu7Pa8zQrgn2ht8bDBaBg1QZ 0N/ecxcCHfxGzXD4satJUEieZhnlf6iS61b/Jn+z7FAqBkoEuIvKDcri7WyoI2av41B102 Lm3IJ3kIhcAN1h37cAD6NbWyAT8eX6soib1shzaZWn9YwB31XPoqFMm2LeD1Dc7spkmUk6 9EYAWrdKTem43zRJD8sg2NlJJyD/ZIZQyTCSsuG7WQo5LL1/PzWCTY6hu+Kh3qyEKzcylj RBnawOUCQkvvci2U5SpmTbB8kGofNTPiZYYvm6ViHiiS+KjM5m4VEy0QXT3L9A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 727C812B00; Sat, 19 Aug 2023 03:20:28 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Sat, 19 Aug 2023 03:20:23 +0000 From: Glen Barber To: Mark Millard Cc: Current FreeBSD Subject: Re: Possible regression in main causing poor performance Message-ID: <20230819032023.GB4090@FreeBSD.org> References: <8A267B4A-1B7B-4D61-AA57-8E3156470617.ref@yahoo.com> <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ZoaI/ZTpAVc4A5k6" Content-Disposition: inline In-Reply-To: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> --ZoaI/ZTpAVc4A5k6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 18, 2023 at 07:09:10PM -0700, Mark Millard wrote: > Glen Barber wrote on > Date: Sat, 19 Aug 2023 00:10:59 UTC : >=20 > > I am somewhat inclined to look in the direction of ZFS here, as two > > things changed: > >=20 > > 1) the build machine in question was recently (as in a week and a half > > ago) upgraded to the tip of main in order to ease the transition from > > this machine from building 14.x to building 15.x; > > 2) there is the recent addition of building ZFS-backed virtual machine > > and cloud images. > >=20 > > . . . > > The first machine runs: > > # uname -a > > FreeBSD releng1.nyi.freebsd.org 14.0-CURRENT FreeBSD 14.0-CURRENT \ > > amd64 1400093 #5 main-n264224-c84617e87a70: Wed Jul 19 19:10:38 UTC 2023 >=20 > I'm confused: >=20 > "the build machine in question was recently (as in a week and a half > ago) upgraded to the tip of main in order to ease the transition from > this machine from building 14.x to building 15.x"? But the above > kernel is from mid July? (-aKU was not used to also get some clue > about world from the pair of 140009? that would show.) >=20 Err, right. I'm confused too, and apparently looked at a different machine but copied 'uname -a' from the correct machine. The kernel and userland are always in sync on these machines. # uname -UK 1400093 1400093 > > Last week's snapshot builds were completed in a reasonable amount of > > time: > >=20 > > root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c ./= builds-14.conf ; echo ^G > > 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/logs > > 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/chroots > > 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/release > > 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/ports > > 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/doc > > 20230811-00:03:13 INFO: Checking out https://git.FreeBSD.org//src.git (= main) to /releng/scripts-snapshot/release > > [...] > > 20230811-15:11:13 INFO: Staging for ftp: 14-i386-GENERIC-snap > > 20230811-16:27:28 INFO: Staging for ftp: 14-amd64-GENERIC-snap > > 20230811-16:33:43 INFO: Staging for ftp: 14-aarch64-GENERIC-snap > >=20 > > Overall, 17 hours, including the time to upload EC2, Vagrant, and GCE. > >=20 > > With no changes to the system, no stale ZFS datasets laying around from > > last week (everything is a pristine environment, etc.), this week's > > builds are taking forever: >=20 > My confusion may extend to this "no changes" status vs. the uname > output identifying the kernel is from mid July. >=20 It is admittedly a month versus two weeks difference here. My fault. > > root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c ./= builds-14.conf ; echo ^G > > 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/logs > > 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/chroots > > 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/release > > 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/ports > > 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/doc > > 20230818-00:15:46 INFO: Checking out https://git.FreeBSD.org//src.git (= main) to /releng/scripts-snapshot/release > > [...] > > 20230818-18:46:22 INFO: Staging for ftp: 14-aarch64-ROCKPRO64-snap > > 20230818-20:41:02 INFO: Staging for ftp: 14-riscv64-GENERIC-snap > > 20230818-22:54:49 INFO: Staging for ftp: 14-amd64-GENERIC-snap > >=20 > > Note, it is just about 4 minutes past 00:00 UTC as of this writing, so > > we are about to cross well over the 24-hour mark, and cloud provider > > images have not yet even started. > >=20 > > . . . >=20 > In: >=20 > https://lists.freebsd.org/archives/freebsd-current/2023-August/004314.html > ("HEADS UP: $FreeBSD$ Removed from main", Wed, 16 Aug 2023) >=20 > Warner wrote: >=20 > QUOTE > . . . , but there's no incremental building > with this change, . . . Also: expect long build times, git fetch times, e= tc > after this. > END QUOTE >=20 > Might this be contributing? How long did those two > "Checking out . . ." take? Similar time frames? >=20 Checkout times are negligible. They are pristine environments, so there are no delta resolutions, etc. Everything is clean from the start. Last week: 20230811-00:03:13 INFO: Checking out https://git.FreeBSD.org//src.g= it (main) to /releng/scripts-snapshot/release 20230811-00:05:27 INFO: Creating ZFS snapshot zroot/releng/14-src-s= nap@clone 20230811-00:05:33 INFO: Checking out https://git.FreeBSD.org//ports= =2Egit (main) to /releng/scripts-snapshot/ports 20230811-00:09:44 INFO: Creating ZFS snapshot zroot/releng/14-ports= -snap@clone This week: 20230818-00:15:46 INFO: Checking out https://git.FreeBSD.org//src.g= it (main) to /releng/scripts-snapshot/release 20230818-00:17:57 INFO: Creating ZFS snapshot zroot/releng/14-src-s= nap@clone 20230818-00:18:01 INFO: Checking out https://git.FreeBSD.org//ports= =2Egit (main) to /releng/scripts-snapshot/ports 20230818-00:22:09 INFO: Creating ZFS snapshot zroot/releng/14-ports= -snap@clone Glen --ZoaI/ZTpAVc4A5k6 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmTgNPIACgkQAxRYpUeP 4pN53Q//bN6NBJwQkG/nuGkJMq6xxnkTpfAb+GMmBewQDZv8Hc+PI+cCZJP6I0FN aQ0D5LDRoeoYjkB5+ZIsX8eaEO+LpEm9eJoUhBJBx/e1xRE7AOA5KDMIo5Ce6Mvq 3oICTZSBQR9CRO6EqJt4oN0x9zx6vOPbNE6neHDIpuaFBVqiFhyzV/S0JfmnRJ0V QNzaCgvEwiN+8UyAY+6ayAqCfBmE6WEs2+ZmbWKRQrwX4n5WsLYgll3pYliEErgS TQRotn5F4fRo9ogNh8qtj3mZsT2jg27zzetvBRQC0zhRiZ6CKSkBxhNnAJ6V5/SW L+P8/N2wyLsL3EeQiJjVcUSkvKrrFmTQ0nQ8eYLD5ppLq5P4z+aXbNy5dmzec+rt p1LcKuX/hLjYjpkP+mtyxEW11z0FPFfxiMt7rI2Z0EzewjiVWFn+pvefQ40rHCB1 7pLWaTMwRrwCMXPtYx3CvQ1uPsTxZAVVdp5RL3qU6Y406EX8m9xxiOKcTDMB3d75 guzJ2d7rbP2VetkDsEgO81zMQ18/TtLDmShMFoGAjSgxLStOM0PCW5J0ZXBxNnZQ bYAyEVD7NMOug/RjfbngrQjflAxDhwlMosj8n47jDNCLdeWaIXZ4q2BQYL7YE43n 7FLxVpziMuh3brdbKm1Aq+3l8EdzFg02elxuoowkvZp3V87ePAk= =pAkC -----END PGP SIGNATURE----- --ZoaI/ZTpAVc4A5k6-- From nobody Sat Aug 19 03:23:00 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSPG2438mz4q3yC for ; Sat, 19 Aug 2023 03:23:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSPFw4pnyz3ZCd for ; Sat, 19 Aug 2023 03:23:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=MmF0bnFG; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692415394; bh=Iwys3/kKpqgIBlnEL7nVpI+hcnJXqrE9Q+j0ys91RW0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=MmF0bnFGD562Pvm0JckNaWvpbM7xwIlsyLc6HVPFTBqNKsdaEhjT/ZzYYKyrMcmDjBgGBvYD4uYPt3Hv4g6WlWcWQJ3kXws1wX8645/DmP1gU14k6Uxav9MoXfUbr2uOKKcPlzeA9Wlo0jeZli2ws6y13nwQpNj3ZeNyKgOqT2u/MB8VaUqE/ixIa1jGyymVDwFLctTljKsJK2c8WwOAKbZRlB9RapQK4RqzcYxhQTr1fL+bwZgzFui/VNCdzIF7ykZcytVopPXhfeTs4BYugsRhkffgBEav4cehVKjFtT7srGQMhBgpxwtBt7ewTN5tU5HhPqpbnXUHgNL+mMDJ/w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692415394; bh=PbUzvKmIQo38G5D84B3YMCsd1zLZYvMPsWF2jLFvQK7=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=gzTZFGuQhX8Ar2AnSpPAfpvXSX694gqKV1Vbr0dtC6Wm4DyEeT4EbqJMFajOhiffdGdCgF9epc+NdAsxW0z9L59wXFolNQNvIsLX6sEPNuVq0MtO+vS/6EIefAUiniSj6PQ5JtMH65Z0rutwHGdxPS2817xtHG/8COmGk1MaYDZnm67zCTnom2p67vjHeFzBpNOBKXjN9I0dE54wyFyWbEi/b7NDCee04W9wIpEePXoXlRekDO8pZyVTS3gZIPtt224FbIqT/F79wuC7au44KzdhtraOZpG48eHv+tLeChIuTG8PGXs/E9kF7p8bOrEPGToflct3Pvx8zweqJbLhzQ== X-YMail-OSG: BLWeMjQVM1kADw3_2VrfAQiVxeickA.nwE2r2gV1O5GY9DFCO1vgccVTKg.2264 INDMGTCvG9ML9Zv96btNcvEBVGly2_PTLKtwTK5dgoVjDNPVbcuW5M2Xf0PfOHYpI0JPxzFJ.0Qx gQARo7oVF1IZ_aunY9mc8huHbyq1UlRO90XQN.kXSNa_zwv.tHa7UxOoU7BYJfQXU.LwSYB.Jrp2 cnEHewXPmN_pM_CjdwuHRPQXhZlp86oF5G_hhyQ1uoQrbQQtUXZ36RZfNNKc5hAOgUHsDLoRtRgm o5r90spPlAN1OEvQaLvHwbFSoHt8hBhg1LCOLboFsa42aUO42AmuLuswwBdcZ7y.P9OJTA17A6SS i90nY07nMrf2Jx2MeVIJVxRJk8ChgzEWwY3HjdzDl.SrzQz7QQFTaP6EH_N6_0Jm7rxAMJ5nG3A8 Xt9aUXO5huS53vXVVSxv35SbB5jK5hpt4JbgnYOB8LlVBO0OTJ9tpxbc1w43rK7fO6CEwlBU6c2N dMPiaG4BqYX4v2f.wqbg0WyEsyKFd80kaBuBqjOnfSFs1kKFV2yPVTvLag7brN_onQMNO5eTw1jU 2ohzZmFOHLD641ktKTJ_dubFFXdLargpoLK4bBBQUJDzP66rTZV0xCHNG1pl.Bsj6JpJjr2JI3Ii Q8Mr7lKhWHSPyqTBfsm.b1XFQG52doJj.hujmPlnTcWU.nraWsJuusQkmwwb8BOsj49hkKnN42bt F2Kvfems9ISEeMGrEztzDBizzati.0FUum_P6uX8vtJxXE9KxnEmyCHdu_.CB88KdUWmHqhag8eo aEhqjN_Q9k2QjNBWlxsN9Xi.n5uuKkne.cHQGFSxRFjMl.B1QRM939iYpvSAdgfkMFcSUXTVelts 83FJH32N5rTk4F.SlLc2fZQ_51erDlYPTaw3xYXRgrZkX9NqzgHzP0kkdN49.Zwnf74CQQx686Cc 5DoQJAj22UuQ5UFEUoDRj9m.ylXP27fh0I8LR62MkF0i2RGPT614knWwtJ08MLbylnCJcIcuMut7 IePZ_RTnMi8Vn91e.sDAXsMAXgoBN3TSaVwVs8KrZpz4xZFWkUGzYnl_qVCuvgxVdOblzg915gr9 jydmWFcs2oFYWbF3t2Tpax6kG9CnVf_JhvOAT6uxxsdbIJtbLFWCcgvmTRhTe6v77RMWqlhinbBr 2Ncmmp28ikWVc60IyQlZkW0wadx4bEugCF4EA2DN2n80HE7hy6wLK7LH1OT3uaMjPbdLZbsUy4.F E.9kov.dZMIaBuslJHHrrh2qAnukq0TR_gSbdpfP1dIiAUyhEcIWy0FLhzWb8oy5zpUZ.He7xd0k RYsvSJVIAiHCh81sQEB2u4N4HFFO9AyBQ.uRNUWaEpLpXnJupmFA39nEb343wP0D3wQLffnqiwVo 9k4bjjQfxXpK8KJ4nMhB9iwQ21p1OO.x4R1unKYJWr_tGVL1Dp13BtnLOM9wLYC07nxzAgMgIpbY l7iPsJ8nd8qBK7EQ_ai84ZIr4.sWRG1OwZPswFW9t7puHS4Lh6R5MO6K2Mz7hqO4.Ka2quy0g2d3 v0NioTLsBCKe5YbAG_FpHmL1S3UtMBKdTqubchBnRPMKV5H8pNgynJ4MXizdWG__2rwXTLxv2ixb zeoLFHt_TcYL6sdJcgD5VXVkvodsNXYoI4gitG3W0tvNTtjSOMzJ1HzfIAfXjWhqfBWIoO7orBs7 44KZrkmZXiza6PNMz3XLE9G33ZrAAumwmQJSKtKmw_N.6f6lSpPeugE76K37y4321Ywn_xpvLBs. Ux_CxbJZ.6h1k3hfP6UuoTg6Ic7AtISgByFQxWD04tJ3YAfKYJX.1zB0gpvER_n_1ij8Q7rBeL7W 3USIbqijOvFjPTLEpaNhACLSf2PxxDX8ysLxDdBI7nb2QMuHDU1du2204.Aw9EdDuq9e.Ql5HGKz MVG61brWuD.MY3YWjR97iT0kZKtbMAFbWhAWk6Bem94aE_FXwCSt0sOpRUY34Q9m8hi55fYbEn.4 Yawwy4q71OTqDDG1R4TzaMJ6Ls7EK5_MHYo0KVS8St5g7bpH8hasfdJw7j.A3J0p60FaFUf7tLHD uncacxpN3Bd4IBuoxLR7U_lnvVsrLOk3g8IFxqM2.WbgpBLSvgfHuLqoJdIpcY3TdAQLnO0.KQ6m bWPw.jCDcTnPCQEOeUbzzV3ZOeKrkVzd28w6mHDCncc0lyeT2BkB41pCBLNCVb4bY3YunlsfQTeg u8GAF93KGEH2.HJQautlGpbCkhGIlHWktctaOMEkbrmU3RydCejuHGmE51xfIf_x5Vcn32x4skg. xt78- X-Sonic-MF: X-Sonic-ID: 04b5ac7b-db92-48f7-9345-d5223bfd703e Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Aug 2023 03:23:14 +0000 Received: by hermes--production-ne1-7b767b77cc-62qqp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 61fec4b730ad27de38541bbbdae322dd; Sat, 19 Aug 2023 03:23:11 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Possible regression in main causing poor performance From: Mark Millard In-Reply-To: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> Date: Fri, 18 Aug 2023 20:23:00 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , freebsd-arch Content-Transfer-Encoding: quoted-printable Message-Id: <1B47E578-693D-4690-A577-947E8C9140B5@yahoo.com> References: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> To: Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; BLOCKLISTDE_FAIL(0.00)[98.137.68.204:query timed out]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.204:from]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.204:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSPFw4pnyz3ZCd On Aug 18, 2023, at 19:09, Mark Millard wrote: > Glen Barber wrote on > Date: Sat, 19 Aug 2023 00:10:59 UTC : >=20 >> I am somewhat inclined to look in the direction of ZFS here, as two >> things changed: >>=20 >> 1) the build machine in question was recently (as in a week and a = half >> ago) upgraded to the tip of main in order to ease the transition from >> this machine from building 14.x to building 15.x; >> 2) there is the recent addition of building ZFS-backed virtual = machine >> and cloud images. >>=20 >> . . . >> The first machine runs: >> # uname -a >> FreeBSD releng1.nyi.freebsd.org 14.0-CURRENT FreeBSD 14.0-CURRENT \ >> amd64 1400093 #5 main-n264224-c84617e87a70: Wed Jul 19 19:10:38 UTC = 2023 >=20 > I'm confused: >=20 > "the build machine in question was recently (as in a week and a half > ago) upgraded to the tip of main in order to ease the transition from > this machine from building 14.x to building 15.x"? But the above > kernel is from mid July? (-aKU was not used to also get some clue > about world from the pair of 140009? that would show.) >=20 >> Last week's snapshot builds were completed in a reasonable amount of >> time: >>=20 >> root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G >> 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/logs >> 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/chroots >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/release >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/ports >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/doc >> 20230811-00:03:13 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release >> [...] >> 20230811-15:11:13 INFO: Staging for ftp: 14-i386-GENERIC-snap >> 20230811-16:27:28 INFO: Staging for ftp: 14-amd64-GENERIC-snap >> 20230811-16:33:43 INFO: Staging for ftp: 14-aarch64-GENERIC-snap >>=20 >> Overall, 17 hours, including the time to upload EC2, Vagrant, and = GCE. >>=20 >> With no changes to the system, no stale ZFS datasets laying around = from >> last week (everything is a pristine environment, etc.), this week's >> builds are taking forever: >=20 > My confusion may extend to this "no changes" status vs. the uname > output identifying the kernel is from mid July. >=20 >> root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G >> 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/logs >> 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/chroots >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/release >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/ports >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/doc >> 20230818-00:15:46 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release >> [...] >> 20230818-18:46:22 INFO: Staging for ftp: 14-aarch64-ROCKPRO64-snap >> 20230818-20:41:02 INFO: Staging for ftp: 14-riscv64-GENERIC-snap >> 20230818-22:54:49 INFO: Staging for ftp: 14-amd64-GENERIC-snap >>=20 >> Note, it is just about 4 minutes past 00:00 UTC as of this writing, = so >> we are about to cross well over the 24-hour mark, and cloud provider >> images have not yet even started. >>=20 >> . . . >=20 > In: >=20 > = https://lists.freebsd.org/archives/freebsd-current/2023-August/004314.html= > ("HEADS UP: $FreeBSD$ Removed from main", Wed, 16 Aug 2023) >=20 > Warner wrote: >=20 > QUOTE > . . . , but there's no incremental building > with this change, . . . Also: expect long build times, git fetch = times, etc > after this. > END QUOTE >=20 > Might this be contributing? How long did those two > "Checking out . . ." take? Similar time frames? >=20 The build process and information is not available. So I looked at something I thought might have a chance of being somewhat invariant and have a limited range of types of (parallel) activity: time differences for the CHECKSUM files taht have timestamps after the last *.img* timestamp, as seen via: http://ftp3.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/14.0/?C=3DM&O=3DD= (so: most recent to oldest as displayed) First today's: = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-26= 4841 1232 2023-Aug-19 00:26 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-26= 4841 1744 2023-Aug-19 00:25 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841 = 1168 2023-Aug-18 22:59 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841 = 1680 2023-Aug-18 22:58 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-26= 4841 1232 2023-Aug-18 20:49 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-26= 4841 1744 2023-Aug-18 20:48 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013= f29d048-264841 156 2023-Aug-18 18:47 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013= f29d048-264841 220 2023-Aug-18 18:46 = FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013f29d048-264841.i= mg.xz 738514252 2023-Aug-18 18:43 So, in rough terms: = RockPro64->2hrs->riscv64->2hrs->amd64->1.5hrs->aarch64 Last week's: = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-arm64-aarch64-20230811-136fc495615f-26= 4678 1232 2023-Aug-11 16:34 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-arm64-aarch64-20230811-136fc495615f-26= 4678 1744 2023-Aug-11 16:34 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-amd64-20230811-136fc495615f-264678 = 1168 2023-Aug-11 16:28 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-amd64-20230811-136fc495615f-264678 = 1680 2023-Aug-11 16:28 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-i386-20230811-136fc495615f-264678 = 1160 2023-Aug-11 15:13 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-i386-20230811-136fc495615f-264678 = 1672 2023-Aug-11 15:12 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-riscv-riscv64-20230811-136fc495615f-26= 4678 1232 2023-Aug-11 14:46 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-riscv-riscv64-20230811-136fc495615f-26= 4678 1744 2023-Aug-11 14:46 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc= 495615f-264678 156 2023-Aug-11 13:54 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc= 495615f-264678 220 2023-Aug-11 13:54 = FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc495615f-264678.i= mg.xz 738475300 2023-Aug-11 13:50 So, in rough terms: = RockPro64->1hr->riscv->0.5hr->i386->1hr->amd64->6min->aarch64 Thus, 2hr vs. 1hr in crude terms. I do not see any 2hr like time frames for 13.2 via: http://ftp3.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/13.2/?C=3DM&O=3DD= =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Aug 19 07:36:53 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSVtg2Rx1z4qMKx for ; Sat, 19 Aug 2023 07:36:59 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSVtf1hJCz4WQf for ; Sat, 19 Aug 2023 07:36:58 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=LMz8BPyL; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2a00:1450:4864:20::62a as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-99d937b83efso203453866b.3 for ; Sat, 19 Aug 2023 00:36:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692430615; x=1693035415; h=content-transfer-encoding:in-reply-to:subject:from:references:to :content-language:user-agent:mime-version:date:message-id:from:to:cc :subject:date:message-id:reply-to; bh=PXdTV+xSvzte1wjpKHbzwo7WxxuZA49V5Y2Zm9WsPmo=; b=LMz8BPyLCYMtkpr/QpoYl1iePDxQvfW0oLMDLySMg6GUMd1HJPRu2ZZrGUjwCNh8Fx btY8VUW8SR8fFIiGLloWhgTQF66/1s7fWEsgreIyRj5x9zkzdB5lkQvqBA43UhDSzYAN gVF8VAWse0bJ9+psMKxyUlnuoQ6p0NCDJFVHyemP7Sq17cTgj8AUZzez19tjyGaJ7u1a +JIVklNHOjcNAp6S0Ue+5L6xXyOUfQ5le6rvX+lJk1YS5hqyj9Y+7hKHQZ/LMSI39Smo iJyBNg5P3Uz9J4b03OSOQ13qvs/I0X4Z5tH8YfrMAOihUnm2C6IRNpCv6ZxzMU+WNh5J VxTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692430615; x=1693035415; h=content-transfer-encoding:in-reply-to:subject:from:references:to :content-language:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PXdTV+xSvzte1wjpKHbzwo7WxxuZA49V5Y2Zm9WsPmo=; b=jFU9JkQVBiTLl7f//niMQno7oBO3m+QTs3OqNAKKvuqiGUAgZSYf2dM7AIY3FFjEF7 zyheO8+kGiGqh1r5qztNX9uVOSU71OjyVLfI4jim8GRU38jYUL2101J0jHfdWFCKnWLr GgZf/rlD8ChoZZem/lyo5jy2VPs/rVoSla9WG0W/86euy9r8gBd2HDE9fU4E9S6rJ9S+ 3iWVZ+hqoV1+ZPE3Ffyo2QzWN9K6kQEzDZJyVDALyB0WQ8IX6tharfKiG0FpMoYvwEfb BBZGipf2tnZC96tHfNj8jgVX7RmAQTpskwdB2agMjlgbIxrs39wylmee6CQf4uBWvNIw 1+gA== X-Gm-Message-State: AOJu0YzddtmhDVxhU93htXQO/35grIMZ7Rz6KD1pAyXPfqXWIRpR2YvN Nr7IjqTHEGkJcxeX0zEY3QAmnWnI6VM= X-Google-Smtp-Source: AGHT+IHofNd7IdLnpJ2G3cSWCAVtxcR3MUgktl1cWc5dsnOaOokXpwFiGBNwv9z391rQlMp5Ioa17A== X-Received: by 2002:a17:906:cc0b:b0:99d:fa95:ecc8 with SMTP id ml11-20020a170906cc0b00b0099dfa95ecc8mr988694ejb.59.1692430615151; Sat, 19 Aug 2023 00:36:55 -0700 (PDT) Received: from [192.168.1.10] (80-42-66-93.dynamic.dsl.as9105.com. [80.42.66.93]) by smtp.gmail.com with ESMTPSA id l3-20020a1709061c4300b009882e53a42csm2229480ejg.81.2023.08.19.00.36.54 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Aug 2023 00:36:54 -0700 (PDT) Message-ID: <9ba2326b-d5de-5faa-edc8-b1cd7235a51c@gmail.com> Date: Sat, 19 Aug 2023 08:36:53 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Content-Language: en-US To: freebsd-current@freebsd.org References: <59FCB309-4A55-4924-98C4-7ACCA70FD299.ref@yahoo.com> <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> From: Graham Perrin Subject: ZFS deadlock in 14: without ZIL In-Reply-To: <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.94 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.94)[-0.939]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[grahamperrin]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62a:from]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSVtf1hJCz4WQf On 19/08/2023 00:03, Mark Millard wrote: > I believe the below quoted messages were reports of deadlocks > based on after the following 2 MFV being in place in their > environments: > > Thu, 10 Aug 2023 > . . . > • git: cd25b0f740f8 - main - zfs: cherry-pick fix from openzfs Martin Matuska > • git: 28d2e3b5dedf - main - zfs: cherry-pick fix from openzfs Martin Matuska > > > … Both were ZIL-related. no ZIL; I never used the feature. From nobody Sat Aug 19 10:31:16 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSZm66xHQz4qXKY for ; Sat, 19 Aug 2023 10:31:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSZm667grz3Kyc; Sat, 19 Aug 2023 10:31:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692441094; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Wt1HEfafFTUPwO3C67/GMoqFdtkJUZljyw+UDgROb3o=; b=sc1GpupEGpjV5NC7v/+MIKLEjJ92bB0xavVK5CsThjVwacQDjsy3PW3r81tsY9BFiIzOTw HjWrgCly7GCd4t+2gDUlFeJZ3rjE3spIERvGvTGX5at17avKB0o8pwmVSES8mFmRYUhTN1 NlhbQDp5bFOqCF8lKRMldhKTWm0MAQGAmqRxwxny9gSm+0TLKtBZScvp+eWAFSTJFdc+J5 xaNoEv9kTfRqeOSdirLF5nuEza76oBUhONaSWeN0SlbegpfGr3WqOPORjhoyQ/eZ6LWRYA iPgIsFpLbxrkvSnjlJ1oBnaxvbQS/Wl+5Yn62breh39EMnnA1O4GmvSRKxgtLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692441094; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Wt1HEfafFTUPwO3C67/GMoqFdtkJUZljyw+UDgROb3o=; b=QnubFkjKcqJYcvzhX+laWZQuF6nrOLkJD6r0tlqD+vdGMtMJJzzX4TVO5NkBCgyYHhsZIa PfLaDfuud/0ZGcoEFUN5PmCjSyrEQQShoAFQb8unJrlqKKZgufvlshEXVHqO9JekHchzyv xNGuB93Au+B8v3MqT4Sohn7AjlsebK1AkLuOEld/lVz+/K43HXHWeQj01U8V8mZyOHp7z2 xHFA1dl6P+Y1WFMAgQJDZeC3cINsOFxjVM9vOQX5pT4rYTA3c2x1hAoYV4tIfRC5+cEtvB /ckcPWKcKuiBwY4EY0Gk+VOyHR3HuVupncFZOER4Z4c+phHwZvn6s5j6ZKRd3A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692441094; a=rsa-sha256; cv=none; b=G1sZbIkleJn/RVQM6enRk7YGT4eZ3BPyapDkkg7V+9qsH5X59HphjzPPG5+EytXH7ORTru x7QQSGd3C4SUlXyXrzqVJ17VY5wR/d8kIHeE9T9fwKoDpNN9UemAJEsOol8lM5uP9jUOkH GReSZk4Tsj0jtmTJagE/GL0PgTARPlV5XbKyj9i4tMr/jhzaLinQyZnTdk9vRXExvzm5pi TIgAfGLxHp+76I3itrIVA2xqHl1Sy05L9Q6FHtCcJhkSmo995MRB/WeMXa684PZwE4m3gO BZlzO/nwF1e0oBG2MjZ39wVfnchkH41SCVBgdyt5hQj4asJuIEzMvb+ifft2QA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RSZm64RbFz7GQ; Sat, 19 Aug 2023 10:31:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id A909566754; Sat, 19 Aug 2023 12:31:31 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_F9250DE6-F7E7-46B9-B25B-455AC1D20BE3"; protocol="application/pgp-signature"; micalg=pgp-sha1 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14: without ZIL From: Dimitry Andric In-Reply-To: <9ba2326b-d5de-5faa-edc8-b1cd7235a51c@gmail.com> Date: Sat, 19 Aug 2023 12:31:16 +0200 Cc: freebsd-current@freebsd.org Message-Id: <23BA44D4-8F35-4A0A-87A2-F765FE6D6865@FreeBSD.org> References: <59FCB309-4A55-4924-98C4-7ACCA70FD299.ref@yahoo.com> <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> <9ba2326b-d5de-5faa-edc8-b1cd7235a51c@gmail.com> To: Graham Perrin X-Mailer: Apple Mail (2.3731.700.6) --Apple-Mail=_F9250DE6-F7E7-46B9-B25B-455AC1D20BE3 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 On 19 Aug 2023, at 09:36, Graham Perrin wrote: >=20 > On 19/08/2023 00:03, Mark Millard wrote: >> I believe the below quoted messages were reports of deadlocks >> based on after the following 2 MFV being in place in their >> environments: >>=20 >> Thu, 10 Aug 2023 >> . . . >> =E2=80=A2 git: cd25b0f740f8 - main - zfs: cherry-pick fix from = openzfs Martin Matuska >> =E2=80=A2 git: 28d2e3b5dedf - main - zfs: cherry-pick fix from = openzfs Martin Matuska >>=20 >>=20 >> =E2=80=A6 >=20 > Both were ZIL-related. >=20 > no = ZIL; I never used the feature. The ZIL always exists, but it can be stored on a separate device for performance reasons. See zpoolconcepts(7), the Intent Log section: By default, the intent log is allocated from blocks within the main = pool. However, it might be possible to get better performance using = separate intent log devices such as NVRAM or a dedicated disk. For example: # zpool create pool sda sdb log sdc -Dimitry --Apple-Mail=_F9250DE6-F7E7-46B9-B25B-455AC1D20BE3 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCZOCZ9AAKCRCwXqMKLiCW o5DuAJ9iKc/jfYMo1yAdw3h6jR0VIOc5ZACgjKEoVL6hYH6m8j3NWKWISg89QHU= =jS53 -----END PGP SIGNATURE----- --Apple-Mail=_F9250DE6-F7E7-46B9-B25B-455AC1D20BE3-- From nobody Sat Aug 19 11:32:44 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSc6k4s4Mz4qbSy for ; Sat, 19 Aug 2023 11:32:46 +0000 (UTC) (envelope-from des@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSc6k4DYpz3Sw2; Sat, 19 Aug 2023 11:32:46 +0000 (UTC) (envelope-from des@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692444766; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EtIcEyAzlVcBi4CVNuEdMvN0OwaHPNiXZWIrFGErU3o=; b=rbNi2RolTCldRjcoumVaCm7tkaesn/gBMqn8zVRYMT+OnV4f/6Mbt3g5w28vVIIVT8a36B DA9ZhIzOYeOSRrP7hEhISjSrqkHE4lPyFXeXSXnUSCx22OgpjO+byfufhJ54ouGb48v31e 7JpOHXbTVvB7gFXMJOFKAXOrZ+RRKXLoQfVR5y+mxsIIn42j2YUM0jX5Ym0LwvA5PRFbg/ 87U9Ex+KNQ7PTxNRuKhX5GMupT3gSVkAdIxze/KfMb5ajxZ/yGrzaGSoYMR2Kg15GR50sn d1bOIczFu7h8J1VbpGzfpusJUpcQhYKHMjNYBFO9oKn0pkScU+nA96kzDc1pfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692444766; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EtIcEyAzlVcBi4CVNuEdMvN0OwaHPNiXZWIrFGErU3o=; b=G/FzqDldEBdn0jqrKPLuQ+EOIw8OfAQeQc9HbFVtejtf0yhMWl/K2XcVwPcDTI9BlObUaM yuTZtza7dnf01Pl3Vu2ewu9FEeF+OcpVWOEJalnOgqZUe57IE+RFvBibQmOu+lO6ssJrer e2DL+TCmCIDcXdQ0hLoaFPnUNkGEwSQiZM8fwOnpuW9eKOA6LenT18EV8ffvICp06DXRoM XJpAufvSs++3Koj+RVPORuNariQZ1+i1ZjvZR9oByM1tApYRVPioT4LZevQL3p8DpSmlGS 2TzsIPOvFTViaKTwkDoxurdyGHDLI7zNWhq75/CMMmySg+cT+s5afO2rIbcnZA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692444766; a=rsa-sha256; cv=none; b=PK0w363ZrYXLgao6CdlH8KjyLr+9hlZUFSj/G1RYw2PaQ7W1WANoihL28VqD2GUDnJjIEM Lc2mBhKx/m/VOLE6PlVekmCRxA+kNNqXclGMIIBhSC8NZyN5KX6e9EqkqARynTA+yACOwD 9A1LH3QQwQK20g36z1lVbzGFOrouELGpQEg0AcEtcB6gAFrOqLQjDSBdOhLXPB+4B56dxU eFkVsyW3zdTu3B4ode15Hl1LbNRMHxhwGYdNBXnqGDAFRsOvSqvr4XnYJ6lFk/I+6NEBpI V8BzDT1BIgBY07IPbFGFrMOtJVyJpjaelS5D4JTqwa7tKNI+b7DVbVpx33rAGg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ltc.des.no (unknown [IPv6:2001:4647:d671:0:36e8:94ff:feca:9834]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: des) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RSc6k2R1Vz8g1; Sat, 19 Aug 2023 11:32:46 +0000 (UTC) (envelope-from des@freebsd.org) Received: by ltc.des.no (Postfix, from userid 1001) id 5EDEC28757; Sat, 19 Aug 2023 13:32:44 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Alexander Motin Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska Subject: Re: ZFS deadlock in 14 In-Reply-To: <868ra8nfo3.fsf@ltc.des.no> ("Dag-Erling =?utf-8?Q?Sm=C3=B8rg?= =?utf-8?Q?rav=22's?= message of "Sat, 19 Aug 2023 00:34:20 +0200") References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> <86350hpiby.fsf@ltc.des.no> <0fd64567-59cd-9276-cc0c-17a09611f11a@FreeBSD.org> <86sf8hnp3k.fsf@ltc.des.no> <86fs4go2qb.fsf@ltc.des.no> <868ra8nfo3.fsf@ltc.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Date: Sat, 19 Aug 2023 13:32:44 +0200 Message-ID: <86ttsvmfmr.fsf@ltc.des.no> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Dag-Erling Sm=C3=B8rgrav writes: > c47116e909 with cd25b0f740 and 28d2e3b5de reverted deadlocks, see > attached ddb.txt. I'm going to see if reverting only 28d2e3b5de but not > cd25b0f740 changes anything. c47116e909 with only 28d2e3b5de reverted also deadlocked, but in both cases it took much longer than previously (thousands of pacakges built, rather than hundreds). I'm now going to re-test an unpatched tip-of-tree (4f14d4b6b7), if I can get it to build. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@FreeBSD.org From nobody Sat Aug 19 11:55:46 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RScdM3n2jz4qckd for ; Sat, 19 Aug 2023 11:55:51 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RScdM1c2kz3WVt; Sat, 19 Aug 2023 11:55:51 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-99c3d3c3db9so229358866b.3; Sat, 19 Aug 2023 04:55:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692446149; x=1693050949; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=T909Rjn/BvSkuNKAX6Nx0/7hwIba6ebRx7EQkoG/H9Y=; b=cQ5D71Z5OpCrLhUa2GzJBjI/cUrcL0K61lG9fHbGqbUst/4RzsfTWxjmfYerLGyOsG 6fwO3vKBkE0sHDq51SFApq8qdWbIPJQs089rlVHiGF2FyOtvs09CAgj63rxmd3ROBoBg uaU8EtkJZ37jGFnxmgcwhNTDnLnO597E5gvIWy5Mn7EqP6JJaP1n2dqTzoQvlZwzSamN /EPYtkLJalcDtY55KupKbu260wQUe2kIi8xeAfoxKCTs2VH0F6yNouo/lTUAH2quMpFL 3YxPQ2+vazYYeC4YPA3DzCFQbSUxTl4eo+g49CZqVxlyQBYusUb5G5RSq2X04JtNBPG7 R0Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692446149; x=1693050949; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=T909Rjn/BvSkuNKAX6Nx0/7hwIba6ebRx7EQkoG/H9Y=; b=NQk/R/EgYTje/DbeLd+wQdrsjwGpSDj0n/+we/L03HCd5Elk26KEIrTjiCdprEXj0W iI/ic0O6JhhWwmKch4aceRludd7Mjj0Uv21ABIk0xWqVWBAz0gaRQ0nz6Z4ylwdvL5DU GsRz7t/eW5HjRw/uzdfVmTwiwqtYPCKpaJkav0a84nYFT6E73kWfCjfICgzM3v8j/870 T93Kmx44lgXPqWSMkvbjI7rLT+mpL47tMyWwzn+3YksgJKufZha6GJZERwz9oUJxIgjG yqOisU3RrvTz7ji5ZeaSVrhSou1bRqSDz7TGQb8DODeiU6kLlXYXbQYLZk7WY8Gyvjdo KOCg== X-Gm-Message-State: AOJu0YxkPQadNoPMHN1zKypaCJ+Vcm1QS7xKb/+WMtuM5fvS0nODzpLP qVuxJ+EZrLpITHPlcOEmZjnh6sPjy/k= X-Google-Smtp-Source: AGHT+IE/mFsSiiihYnJW+8L3WH0wh9NZwPbND0dfQNXIwIQwiA90aLkgmPMm7fw3sedOvXdzCO0QSg== X-Received: by 2002:a17:906:2099:b0:99b:64d0:f6c8 with SMTP id 25-20020a170906209900b0099b64d0f6c8mr1614043ejq.50.1692446148597; Sat, 19 Aug 2023 04:55:48 -0700 (PDT) Received: from [192.168.1.10] (80-42-66-93.dynamic.dsl.as9105.com. [80.42.66.93]) by smtp.gmail.com with ESMTPSA id i6-20020a170906264600b0099bd1ce18fesm2626389ejc.10.2023.08.19.04.55.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Aug 2023 04:55:47 -0700 (PDT) Message-ID: <174925b2-e8ba-2da0-7dfd-d062e303c53a@gmail.com> Date: Sat, 19 Aug 2023 12:55:46 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Subject: ZFS deadlock in 14: with ZIL :-) Content-Language: en-US To: Dimitry Andric Cc: freebsd-current@freebsd.org References: <59FCB309-4A55-4924-98C4-7ACCA70FD299.ref@yahoo.com> <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> <9ba2326b-d5de-5faa-edc8-b1cd7235a51c@gmail.com> <23BA44D4-8F35-4A0A-87A2-F765FE6D6865@FreeBSD.org> X-Priority: 5 (Lowest) From: Graham Perrin In-Reply-To: <23BA44D4-8F35-4A0A-87A2-F765FE6D6865@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4RScdM1c2kz3WVt X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] On 19/08/2023 11:31, Dimitry Andric wrote: > On 19 Aug 2023, at 09:36, Graham Perrin wrote: >> … >> >> no ZIL; I never used the feature. > The ZIL always exists, but it can be stored on a separate device for > performance reasons. See zpoolconcepts(7), the Intent Log section: > > By default, the intent log is allocated from blocks within the main pool. > However, it might be possible to get better performance using separate > intent log devices such as NVRAM or a dedicated disk. For example: > # zpool create pool sda sdb log sdc > > -Dimitry Thanks! I looked at the page umpteen times, that part never sunk in. Now I know. From nobody Sat Aug 19 18:13:36 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSn1H5VR8z4mHms for ; Sat, 19 Aug 2023 18:13:39 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yb1-xb2f.google.com (mail-yb1-xb2f.google.com [IPv6:2607:f8b0:4864:20::b2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSn1H3pgdz4Tyc; Sat, 19 Aug 2023 18:13:39 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yb1-xb2f.google.com with SMTP id 3f1490d57ef6-d712d86eb96so1881204276.3; Sat, 19 Aug 2023 11:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692468818; x=1693073618; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=cviUJMFFwVe3uSD9umiwERFgrMea1UuamcwpyCOeQG4=; b=I1awdNiQh+Z7GLPArNfyJf+oIowxUo7bWDlfxVNgt0vfkfEIHNF5qJHBAvO6JFpuzC e5pKaOwEXtVuYaVq3TDaoMvO3g3YQBuit2WP//BfULOhc82KXv60nLfIZ4bpapry4shX ysNv7V/at6E/LnZcIS48SFli5QB7+nnkNMAZKfHihCAWr95zPECICz3soaFsikD9y6Bj RNqD92LQ8UJm7ytH1T60WYOo1NVtGZYLBlKqvK9tLzEAi/FC94O7y+atz5l+tEBp5lKE MuySNL2hgqeFkhNKeA6+N8wBjdAmrUeUnYuxaserUj2Do40DGRPMx7xx7Nz1xDRtdozX hIXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692468818; x=1693073618; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cviUJMFFwVe3uSD9umiwERFgrMea1UuamcwpyCOeQG4=; b=lXul4t/0hnPBGMCqcbpti5Q5gOqGonUuuY6HDa+lkKjsoTs/dW3Y5J6HIUhhf8zbSV XzMymwLOE6FwCVvZCv0BjfmRLSIquQBM9UxTIZ/fML55Ep2iw3GSo6U054f00Ae7pU9u pEzcI+00e5A932VMKGocYtgAVd3l9TrfAGrx0GRq1v2Y2KNVQ/wufTab81wI8SXZP2PC ciOYmMAHNaiid0Hl+ULN3V/TXjOqBEK86R+GM1oO4nqb8SMzA9+CSyQY44KZPj6laEmf 6akiOP4GwvGiVhj6RfB9llmq0LYKKmtWOv3yC6AhPse0wTdWNLT4yBr4Ta9VhIuRFIX+ ca9w== X-Gm-Message-State: AOJu0YySCFyV577kttIRez1tQ5cdk76ZRowjvMdwh5qobbD0N85MN8x4 mEEy5Sj+yCQ5ArYxdZv49Qzz8AKp1rU= X-Google-Smtp-Source: AGHT+IFwYG3Yj9IBIqR/nrd4CJ4QkGw2r6mBhAyxmx2kC0W81/KlIWZbnFGlx2ojX1A2MAlqpkwDdw== X-Received: by 2002:a25:d149:0:b0:d55:370b:aa2e with SMTP id i70-20020a25d149000000b00d55370baa2emr3191541ybg.25.1692468818276; Sat, 19 Aug 2023 11:13:38 -0700 (PDT) Received: from [192.168.1.66] (104-55-12-234.lightspeed.knvltn.sbcglobal.net. [104.55.12.234]) by smtp.gmail.com with ESMTPSA id e191-20020a2537c8000000b00d6861b49f01sm1073405yba.65.2023.08.19.11.13.37 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 19 Aug 2023 11:13:37 -0700 (PDT) Message-ID: Date: Sat, 19 Aug 2023 14:13:36 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: ZFS deadlock in 14 Content-Language: en-US To: =?UTF-8?Q?Dag-Erling_Sm=c3=b8rgrav?= Cc: current@freebsd.org, Mateusz Guzik , Martin Matuska References: <86leeltqcb.fsf@ltc.des.no> <86h6p4s64h.fsf@ltc.des.no> <86a5utrafp.fsf@ltc.des.no> <86350kqokl.fsf@ltc.des.no> <86y1icp95t.fsf@ltc.des.no> <86ttt0p8wv.fsf@ltc.des.no> <197ead1e-210a-6be6-7e24-5c56b14bb777@FreeBSD.org> <86350hpiby.fsf@ltc.des.no> <0fd64567-59cd-9276-cc0c-17a09611f11a@FreeBSD.org> <86sf8hnp3k.fsf@ltc.des.no> <86fs4go2qb.fsf@ltc.des.no> <868ra8nfo3.fsf@ltc.des.no> From: Alexander Motin In-Reply-To: <868ra8nfo3.fsf@ltc.des.no> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4RSn1H3pgdz4Tyc X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 18.08.2023 18:34, Dag-Erling Smørgrav wrote: > Dag-Erling Smørgrav writes: >> Plot twist: c47116e909 _without_ the patches also appears to be working >> fine. The last kernel I know for sure deadlocks is b36f469a15, so I'm >> going to test cd25b0f740 and 28d2e3b5de. > > c47116e909 with cd25b0f740 and 28d2e3b5de reverted deadlocks, see > attached ddb.txt. I'm going to see if reverting only 28d2e3b5de but not > cd25b0f740 changes anything. Yes, it looks like ZIL-related deadlock: ZFS sync thread in zil_sync() is waiting for allocated ZIL zios to complete: Tracing command zfskern pid 5 tid 101124 td 0xfffffe0408cbb020 sched_switch() at sched_switch+0x5da/frame 0xfffffe04090f7900 mi_switch() at mi_switch+0x173/frame 0xfffffe04090f7920 sleepq_switch() at sleepq_switch+0x104/frame 0xfffffe04090f7960 _cv_wait() at _cv_wait+0x165/frame 0xfffffe04090f79c0 zil_sync() at zil_sync+0x9b/frame 0xfffffe04090f7aa0 dmu_objset_sync() at dmu_objset_sync+0x51b/frame 0xfffffe04090f7b70 dsl_pool_sync() at dsl_pool_sync+0x11d/frame 0xfffffe04090f7bf0 spa_sync() at spa_sync+0xc68/frame 0xfffffe04090f7e20 txg_sync_thread() at txg_sync_thread+0x2eb/frame 0xfffffe04090f7ef0 fork_exit() at fork_exit+0x82/frame 0xfffffe04090f7f30 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe04090f7f30 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Some thread requested fsync(), allocated zil zio, but stuck waiting for z_teardown_inactive_lock in attempt to get data to be written into zil, so zios were never even issued: Tracing command blacklistd pid 521 tid 101136 td 0xfffffe040d08d000 sched_switch() at sched_switch+0x5da/frame 0xfffffe040c25c710 mi_switch() at mi_switch+0x173/frame 0xfffffe040c25c730 sleepq_switch() at sleepq_switch+0x104/frame 0xfffffe040c25c770 _sleep() at _sleep+0x2d6/frame 0xfffffe040c25c810 rms_rlock_fallback() at rms_rlock_fallback+0xd0/frame 0xfffffe040c25c850 zfs_freebsd_reclaim() at zfs_freebsd_reclaim+0x2b/frame 0xfffffe040c25c880 VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0x35/frame 0xfffffe040c25c8a0 vgonel() at vgonel+0x3a9/frame 0xfffffe040c25c910 vnlru_free_impl() at vnlru_free_impl+0x371/frame 0xfffffe040c25c990 vn_alloc_hard() at vn_alloc_hard+0xd3/frame 0xfffffe040c25c9b0 getnewvnode_reserve() at getnewvnode_reserve+0xa0/frame 0xfffffe040c25c9d0 zfs_zget() at zfs_zget+0x1f/frame 0xfffffe040c25ca80 zfs_get_data() at zfs_get_data+0x62/frame 0xfffffe040c25cb20 zil_lwb_commit() at zil_lwb_commit+0x32f/frame 0xfffffe040c25cb70 zil_lwb_write_issue() at zil_lwb_write_issue+0x4e/frame 0xfffffe040c25cbb0 zil_commit_impl() at zil_commit_impl+0x943/frame 0xfffffe040c25cd40 zfs_fsync() at zfs_fsync+0x8f/frame 0xfffffe040c25cd80 kern_fsync() at kern_fsync+0x18a/frame 0xfffffe040c25ce00 amd64_syscall() at amd64_syscall+0x138/frame 0xfffffe040c25cf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe040c25cf30 --- syscall (95, FreeBSD ELF64, fsync), rip = 0x3f979eeb074a, rsp = 0x3f979a449c38, rbp = 0x3f979a449c50 --- Third thread doing zfs rollback while holding z_teardown_inactive_lock tries to wait for transaction commit, causing deadlock: Tracing command zfs pid 65636 tid 138109 td 0xfffffe0438d721e0 sched_switch() at sched_switch+0x5da/frame 0xfffffe0439b2b950 mi_switch() at mi_switch+0x173/frame 0xfffffe0439b2b970 sleepq_switch() at sleepq_switch+0x104/frame 0xfffffe0439b2b9b0 _cv_wait() at _cv_wait+0x165/frame 0xfffffe0439b2ba10 txg_wait_synced_impl() at txg_wait_synced_impl+0xeb/frame 0xfffffe0439b2ba50 txg_wait_synced() at txg_wait_synced+0xb/frame 0xfffffe0439b2ba60 zfsvfs_teardown() at zfsvfs_teardown+0x203/frame 0xfffffe0439b2bab0 zfs_ioc_rollback() at zfs_ioc_rollback+0x12f/frame 0xfffffe0439b2bb00 zfsdev_ioctl_common() at zfsdev_ioctl_common+0x612/frame 0xfffffe0439b2bbc0 zfsdev_ioctl() at zfsdev_ioctl+0x12a/frame 0xfffffe0439b2bbf0 devfs_ioctl() at devfs_ioctl+0xd2/frame 0xfffffe0439b2bc40 vn_ioctl() at vn_ioctl+0xc2/frame 0xfffffe0439b2bcb0 devfs_ioctl_f() at devfs_ioctl_f+0x1e/frame 0xfffffe0439b2bcd0 kern_ioctl() at kern_ioctl+0x286/frame 0xfffffe0439b2bd30 sys_ioctl() at sys_ioctl+0x152/frame 0xfffffe0439b2be00 amd64_syscall() at amd64_syscall+0x138/frame 0xfffffe0439b2bf30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0439b2bf30 --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x1afaddea3aaa, rsp = 0x1afad4058328, rbp = 0x1afad40583a0 --- Unfortunately I think the current code in main should still suffer from this specific deadlock. cd25b0f740 fixes some deadlocks in this area, may be that is why you are getting issues less often, but I don't believe it fixes this specific one, may be you was just lucky. Only https://github.com/openzfs/zfs/pull/15122 I believe should fix them. -- Alexander Motin From nobody Sat Aug 19 18:40:59 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSndB3mqcz4mKKF for ; Sat, 19 Aug 2023 18:41:18 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-55.consmr.mail.gq1.yahoo.com (sonic308-55.consmr.mail.gq1.yahoo.com [98.137.68.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSnd868JQz4Xp7 for ; Sat, 19 Aug 2023 18:41:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=ZrlW0Egb; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692470473; bh=bUgIEGa41pnqkDwg4aEaM4HPHHGGv5WOy4pjJRDm+XQ=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=ZrlW0Egb1rp/jNevygAZT6pCU3EEi0uK1HKCEK9x7pG/vvg2kWYxcM13/6ktQwv3zXtEWuw5SJCnkeWyDNz/NKMkN3cVbsL33+2YAb4CIlbW4xKeUG9JqJfw1fnFUPCkDS4v7Asy6FvNGW1kkO43f8PqmTBKDUvnMO7Mdm+aFXxrOXc1tpRK+qRC1s2eBQBi+VAWosUPtf5uyDUeExgZptDQpnUksfNttiFTzvYHgN/GiEkjBnZ4bOlrIg8EAN5SMeEv8AA3GfPA5NSYxj0vTNsHUk94+eExpYuQzZumeY2WS3O1W8LrRNSuUME2OWUC/yx9a8qCMYtSVQXh6idP/w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692470473; bh=H0j6+dCLyhIS2fOlCaUJYdqkNOvLWZ+wYMCXtMOFkJI=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=rZVC5/SiffYIu2wR9hwTJ7DLVhsYiAFYFcI0I7PLKhP9xqtbI9X4lMGraDcxkQVOWnzX4ckJPQnU9EEXjqXqa3wD0fU+16QlTZyZiRxC6EDChQlhYsKL40H/PYvF5gHrgV7VfJ6G8WfT4aUS4g7goC/pp1gxvRxfUgyo2anUPle/rXKSAgj/3F5D3gMISKacgg+4Vs4wGk4eNaiZiWx9IfdOcdlNQR6vJ3eLVt6OtRP9vHZtWBFnLYaQ1Ov0A5pGkiNyBOeKHE9OEmTMobSgkYfQp0rwiZGL09NM6sySLUAHujANAxZAHTIhIl2oYLxVN60DLXYTCS6GVrjZma4CsA== X-YMail-OSG: 0R5ALFsVM1kU1qDx8p0SE_ml4s8X1gwlLFLmBq.wDIz.CYgsb88LbcV9XUqiYu3 wwqmU3LUGzDFbY5MoCIaRm4qrpa6uWopw83NGk6QP0FaC36f0kwEWZD9SY7p.lx5uHsbO1VCwxGj 12t21PmTPsvrfJyYMcuZ76gm3XYjA75.rQCugA7UpkRVnDwTk_yiv4GanuwxwFaiM2dZWSiT2A1v Jk8q9b3.n0DvIeW.CTE.sorD4hhobbeif7_Oluc8XfcVN5NfvlSvVt49OqdmPO2E0J0jSNNRXKts 8Kf5GcgYDi66MvyjHSFU1w21.og8sSWvCGBu1q1HWLHZgo.cNNBdf6XSwnvziXtk172b22nfJ9O0 j64Dhm4yRq6QzusVc2mMjti6_cMzfRIfU.JJ1KjOciiibzRDFzOZuYaEOFvdPkXB8w3UGikVl.2M Q2diKElgvP2VI5S4BCirp1JgHBK02fQw.zNXRqBFBAUSc8MCG3H70D3pFtyBKCHphSyodWAaQEp3 s7TX_bOfrVhX6YxjhqSgp_3tjXlyV5.3y_D_e2.fVEqsxyRsZcxZVTbieWYjTprSs3rl5YXwOscI C0K3NxlFchK_yKrE_exYXhpZqTtA5NrenTPvtQofF7cLX8cBKZlTl5ld9QXOLM7LQDMZWrTauyjp BG1_NKnJkgIq5vUUYjfnmeAtA2_7eJhUnyQAfBuqMbPZUBUd8UJL3VKoRa4FJnhf7LrcW8Tsrp0z UZxHxacngeVWaN98AizOfiP4XWR3JQlLMR_6bRqzxw9Ki1fA1g3dxZd1FPoQ15Hx0rj4ROkY5um5 jRSJiHzbBQE8zuxFBlrK2L0OZgDEXn.hYNkGO5Tks6N79B.HCfJckJs5lueTpr3RAfju7sLb_ZWR wGBDExbfmki3myC1IoLNVQPlOhCKrDhTUoGXRNMpdB5eYFsp77mWb1O52Jlw2GnFjUhTlDS3YIAB e1bUYM8neW86bN._Rb8ZXw0tFpDwLmQahEftI11W5bTJe0M9UTxuPAEF260rT1hoJiV79mf8IJQs wIG.06kREnPLBj5qaeXi68aGJLP6niyoxf7LjWVaL6W_LQonIzTAhAOOJGHg7vDQtMBTp8f2lMtO jZYNFSp9qjCy9KhiURyYMmBic5Hi1n.dQgyq6XwyX0NEeHuCcFuUefP3cV5zm7v.edBFjWgQ8O7x e9HL7TbouUF1254Dbf1.JGte8VyOltmHuoQSxLGYKNX49459ib43UUHP7O2HnwtdgO0ykbii9Gv0 csm6z_74zYJ4I.VSxgDGwI4twfaYIyBggKBwFegjGZdppgA8Z2s_YuqvJmlpic3VkCJNWQHguQkm QS8alIdNygvxwPVxta19CoKA80bT9JgRd59VwODaPwim5vOm.XgOEk3YBLLcQ2ceh6Gy77Nk.VaJ DgO4zCEeor_RTDuD1ukPckhnPUG0WQCNGMEy5xy29O4rMPvMFH.GmqaBm6cpYeLZ6fDvpiVByA2F NlvWMqXFGXncpRpi9J.qKjy6EszsVpqxlydp6ThQxBJkWdy1CQ9Yo_aytQkN5ZRDKbBErJ8jupQp rREAbnarhRgjIRLfiHmGNE71x9kJWTJhsFqlUrC4Zax.XZbx7zpt8ixUxcxzEepWcLAsUnjPpPer 1Mb_cv4z6gax94i4oLptk7O.T.iAA93Vk1EVqaoJI4CpUHUITX9xfeS4EXxwTwXz9OOR2SSGqjUy BFFanFotMkeg3VRJJPrTzPEJVBhd8Zoo4KLwLGEf5Ba.fIK7DILPbFcbDhtTDv6Nla_8nTv94kig s8f.x5EHHxqjwTiP3tO_A3Ehx4a_I75CLzYwM50kOMfS0uV4bXgh6T786zTuPRR8w.vsFAVuFFHa ZJcbLugnm00ww.QK2feDvSNSxNJ82Z2jqEd8NIem1rerxnurY6_zy8l5ihbsMdEQozVUPnNN4i_J GcShtsXu7qVxNXAXi.ElKLHkpFmtLrcAwoLyNtc4REuQFRs.50sHLqcnFhxtUvpJ2mQx4aD2yxr5 2oW7K6Yob2HEAqETh.ehiRMCHXg.UJzTpuxCrcd9SimpquO77z_rC_QF.e68WLDRohM83iZlnuiw .laK56lOW8DYi_FftowJdEpj1xcnYUzi1QBTvK8CQUqdtUxjx3e8XZMHFizBf6vUiFBC.ozlZAsV 265K0Eh4JJ9oBeDMwW9jt942TL44hszTMh1Z0Ztrg8mCxcwDZAw5ZOXg3rdS4r96mVpQwv3UvNgQ 2YoobT8spNic50yqcHNEhHZ.ZspAJSJP7ORHwd6tcibnEAsxbHH0Y0.CoBxQEBg-- X-Sonic-MF: X-Sonic-ID: ac5d9cee-ca32-4303-8a20-fe4b326e65b3 Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Aug 2023 18:41:13 +0000 Received: by hermes--production-ne1-7b767b77cc-7tm2h (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 0a0aaeab01d107c73ddc8b75c91cfc1f; Sat, 19 Aug 2023 18:41:10 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14 Date: Sat, 19 Aug 2023 11:40:59 -0700 References: <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> To: Current FreeBSD In-Reply-To: <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> Message-Id: <0F2C42B4-36FF-443A-A174-5B0CC57C4FC7@yahoo.com> X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.31:from]; RCPT_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.31:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSnd868JQz4Xp7 We will see how long the following high load average bulk -a configuration survives a build attempt, using a non-debug kernel for this test. I've applied: # fetch -o- https://github.com/openzfs/zfs/pull/15107.patch | git -C = /usr/main-src/ am --dir=3Dsys/contrib/openzfs - 13 kB 900 kBps = 00s Applying: Remove fastwrite mechanism. # fetch -o- https://github.com/openzfs/zfs/pull/15122.patch | git -C = /usr/main-src/ am --dir=3Dsys/contrib/openzfs - 45 kB 1488 kBps = 00s Applying: ZIL: Second attempt to reduce scope of zl_issuer_lock. on a ThreadRipper 1950X (32 hardware threads) that is at main 6b405053c997: Thu, 10 Aug 2023 . . . =E2=80=A2 git: cd25b0f740f8 - main - zfs: cherry-pick fix from = openzfs Martin Matuska=20 =E2=80=A2 git: 28d2e3b5dedf - main - zfs: cherry-pick fix from = openzfs Martin Matuska . . . =E2=80=A2 git: 6b405053c997 - main - OpenSSL: clean up botched merges = in OpenSSL 3.0.9 import Jung-uk Kim So it is based on starting with the 2 cherry-pick's as well. The ThreadRipper 1950X boots from a bectl BE and that zfs media is all that is in use here. I've setting up to test starting a bulk -a using ALLOW_MAKE_JOBS=3Dyes along with allowing 32 builders. This so 32*32 or so potentially for load average(s) at times. There is 128 GiBytes of RAM and: # swapinfo Device 1K-blocks Used Avail Capacity /dev/gpt/OptBswp480 503316480 0 503316480 0% I'm not so sure that such a high load average bulk -a is reasonable for a debug kernel build: unsure of resource usage for such and if everything could be tracked as needed. So I'm testing a non-debug build for now. I have built the kernels (nodbg and dbg), installed the nodbg kernel, rebooted, and started: # poudriere bulk -jmain-amd64-bulk_a -a . . . [00:01:22] Building 34042 packages using up to 32 builders . . . The ports tree is from back in mid-July. I have a patched up top that records and reports various MaxObs???? figures (Maximum Observed). It was recetnly reporting: . . .; load averages: 119.56, 106.79, 71.54 MaxObs: 184.08, 112.10, = 71.54 1459 threads: . . ., 273 MaxObsRunning . . . Mem: . . ., 61066Mi MaxObsActive, 10277Mi MaxObsWired, 71371Mi = MaxObs(Act+Wir+Lndry) . . . Swap: . . ., 61094Mi MaxObs(Act+Lndry+SwapUsed), 71371Mi = MaxObs(Act+Wir+Lndry+SwapUsed) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Aug 19 19:18:42 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSpSj3l8Mz4mMMf for ; Sat, 19 Aug 2023 19:19:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSpSh26Vvz4dM2 for ; Sat, 19 Aug 2023 19:19:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=szrMRsNh; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692472737; bh=VOHbdOGh7883w2s+W48/zWYhaclTBlP4JRm4jgHhPew=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=szrMRsNhNWQssKD+GswcVrVzg2UIQA3Oegp8DVE+WSfJ+Vm+TN4Pmqi51qZpE9AsXWRYhU47+IS8Z58LqbZ7X6NzTmpn0mVHvrx+owWUj03xtnG6y1NOgaptejeP1BR/Efj0A8MGG183Jd1aapbYo6IoDM1v+SsQ4BlWgF5PSK3PS2E6i7Z8stuIFrSH4nCiQ+gFoJ2MU/x28i7RroIHl32KKWibY6O4T3toiBkg//VNt57aoFYz7SqMyL25a6y32uX3pCXoh1ws2RzhMK0OQ5YH/xqb0znoTeoMSfroYtMSJdkOMatM7h/ty45WJzpbd7uYWAYSQUgSUnormxr4eA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692472737; bh=Jio29q0KzfbO+nE6HkORXnW3a9SpzxNJY4OLTbMVNpc=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Ee9WVmSW63rAqdjMHxCCpPiRkDUlZDHkU+EnhNB3nNU1hUPlMtaNi8+lHki3kD9C1uYzr1wj2WJC6bJ6nWE6Xh8SAxLE87SB4po5ixQUZXecEIbhFhHXttLwr4tUs6nNjh7lxj5M+O+Ags9VNOdP9nO4zxaZRqWCHYrh9fegZ4VnGey2p/pia/IMpqNAazGLW8RK1RbIieF50BIhSYZ5Cs8PawPCwhywfGTxZ38exugcwmTbNZAhU068Q4Yz/5ywU7PsxQIqM7Od56XWFYcKheteNa4vQ4b1AN07MUZbH/RZH/NboX8yQDpHEZO7ENAzravPFwiaO7XkY7dH0LNOhw== X-YMail-OSG: s.8hsuUVM1ndF2lCC5rKBk8tPhbA9dsw.HKKXGa1SegC0SxnjP6ZWxZ2Lw.yiWk uYZhTfHEAhAew8WSXL8eUPya35gHMcOJ39Z0lQoISrAiZR9aXiANmTPJkO_PTTldm5QLgPYUZBry 8_g5XwXerLx5U6rvDMxImZrjQ_ayxg7omPrcMQr5PUz_C8YKMl6k_utHarONARaGrPbO5j8ryxpE ox5sEw5j7E0KxkiZ_DGnq0HNDM9czgxb5b6JAak9qZc5ryo3HaCQG1XByKXi.dBYdt5.qS3svvHy Ivd6gdV1Iemu.LcHqVS7D4LESM10dlRFWtKfysXcg.LbTIOoq4F0d6HgYgmFeapCQbrcTeHiy61G mkz6t8h_aERGgCKMnOY3gmx_R9BRtQd4Besd2IolrClDq2tmwKTMiQ0pfIXLu6i96v7gW9IT33g8 LodvUt0K4BJYer5OxmujRO3BHoF6a5C0OO4mBOYV15KTIGkqpWp7uTfdVSc58Rp592cW5DjEHhvM iO.DNPIVSss0ORZ.KX2bVBObRxMKDFLSej7nxv7Ggv4y5r5t644wkzw_wSbe3AoobGjbXemLfzjE KoxV61CtvlFzmA1dTHykwaWOijhhPB.7mhbz3oX0rR80FM4WAnHSU2bw2cMNOmUxc3vXh1uFf4Zf ovCXk_o8i9UxpeyKUHjWKx.6vcbUXQnJhMfDnaAsKfA8vSrXAHC2YbNAFtZmNoWFQ9emPpFzorjV wL4His6NsxninbMoN879PUkjkVs6iDzZVo2JRGO0PlS27C6FZNtKJrUfmMPeo057X7o_TC5_tJgV 46Av.RjZz58n4HFyyL7BV2kuWTjCCqzTDqaeUv7haoHTELq6M2jwdU62UZOCOHQjTfj5rDLMw3_z xaitF5AK6H3fPgjRXm2MDgvKj.JcyosuGvPCHtsv9vtPUjKA1L8Im6_VY7STIT4_MWr8cjc4ay95 V1DDLZ_erJJnSXIXsiAZcatFeKTxDZoJAYa8QECrvQN0ryv1i9q2ZQdO3kRDzU6CmxHQCR8rO6FC Hk5m18q85aT91zMOlsKx2DRrxoEHkr7getxWiJRSOgstaJIrEw.pGGZykYexf1aE4vByJzhh8wv1 6Ekfxe4fro702WGHILd3IRzkokt7VZGOOJEnORQAxqYQxwdmqnGdCM.o16XN3kY3D8MG82pqf8sg JT0qHRt0NhVLGa5hXPf3mAM2s97DRY5sV9rQVS2KHeKwyj0mE8y.uq4wvC0O65PoQrVh5hNIzF1h _SUyrAaycgosdhdHDfPJK9wqUE8gfOQzL_AbkSOMun0zPgHWoG7mYJCTrkUAHYfuDKg2OL4JjFEY BLVJNFFmhDtsmMSzXea1gSi3pGA5CP1Ze0GZN0gHK48TQ9wbzyplUe.e03eL4KU0Vi7uabaqDEv2 IjNa5Yjf6q3CKIHBuB5u4vr0UpyWrx2x9x9zNXTkOEzi7fknBuBiDLrb2hyOXpuUtOGms0_kBRmz JuJYeTiojydcfBou3yzNG1obkCiITy3pOgc0fSvy6uwfBT16eFfnB2WoJEHHUhUiitifbeFnk0KX PLtwKBmj_vB7CWqfPivsqV.3_L7EhNEVYr7zDHEga1b.TdK5ywSYwG2XvxEQhnIZiDVS.umyZojw SR15DCAxhs588y.9uEEtUcMkfdg2DFgynofN3CRtBAbimFV949aafGXx4S4g4tjDYITm5acnA0ff I7X2LfhOkE8E6wSoxTLlWG6KjWsSr9vMaf7TWRSDzDii5nDxi8Pp4QlrdsWCex8jGgH2x8iHATTp lf14Ke34sJxRXpzPDUO_4l9.05rxGNwU.VD3RWHGo_ojckxeSpWQqD1T.wBnSMYidAfEXWFJOmMZ V1tIk3dsKEu5.ktpQ1_ZmnYdobP04xXBANTm95mJvBZtMhilpOfyDvDa5DpShl_JwoVnDOXoE6jj V3sIEVAWujvzkHNvNy6tJ62yZT.B9eUm.5Gpi9qUVSoNA5xKqfWIR2B8qijR8.feI9Nb5I4SCrmw ezMaLbNsu1Xf2qaFhMxqu_nAEZAFHmWEBP8WTrHMVkPEt_BB8J6az3xHQmhsgMpqi78xA1eoa4kx BAZZ_s5J.nrLq6GQzVFnpCZuZRpK0ez_J5Y3o0bOZ.tQBrM8cdQuS819coaOj8.JN2mECVuc0S.r PmmrNdA2WWohqlEo_i8eZRVOphkYFdL3VvIbbjDRHHXVtLao3mdLLxbh3RhoaKqy_Pu_twoCuBLB K_cAR4LyJpG9HpdDHgxMAFlfrXdoDYdWFRZOyUBTju5zRAmMZAiv0fSBipI1x X-Sonic-MF: X-Sonic-ID: bbf5009a-e1d0-4086-8703-503483729189 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Aug 2023 19:18:57 +0000 Received: by hermes--production-ne1-7b767b77cc-c8mvb (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2f951008d3317f83f9ccb5871b5c6d61; Sat, 19 Aug 2023 19:18:54 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14 Date: Sat, 19 Aug 2023 12:18:42 -0700 References: <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> <0F2C42B4-36FF-443A-A174-5B0CC57C4FC7@yahoo.com> To: Current FreeBSD In-Reply-To: <0F2C42B4-36FF-443A-A174-5B0CC57C4FC7@yahoo.com> Message-Id: <3AA253E3-C4F0-4AA3-9C37-D77E7527A458@yahoo.com> X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.83:from]; RCPT_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.83:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSpSh26Vvz4dM2 On Aug 19, 2023, at 11:40, Mark Millard wrote: > We will see how long the following high load average bulk -a > configuration survives a build attempt, using a non-debug kernel > for this test. >=20 > I've applied: >=20 > # fetch -o- https://github.com/openzfs/zfs/pull/15107.patch | git -C = /usr/main-src/ am --dir=3Dsys/contrib/openzfs > - 13 kB 900 = kBps 00s > Applying: Remove fastwrite mechanism. >=20 > # fetch -o- https://github.com/openzfs/zfs/pull/15122.patch | git -C = /usr/main-src/ am --dir=3Dsys/contrib/openzfs > - 45 kB 1488 = kBps 00s > Applying: ZIL: Second attempt to reduce scope of zl_issuer_lock. >=20 > on a ThreadRipper 1950X (32 hardware threads) that is at > main 6b405053c997: >=20 > Thu, 10 Aug 2023 > . . . > =E2=80=A2 git: cd25b0f740f8 - main - zfs: cherry-pick fix from = openzfs Martin Matuska=20 > =E2=80=A2 git: 28d2e3b5dedf - main - zfs: cherry-pick fix from = openzfs Martin Matuska > . . . > =E2=80=A2 git: 6b405053c997 - main - OpenSSL: clean up botched = merges in OpenSSL 3.0.9 import Jung-uk Kim >=20 > So it is based on starting with the 2 cherry-pick's as > well. >=20 > The ThreadRipper 1950X boots from a bectl BE and > that zfs media is all that is in use here. >=20 > I've setting up to test starting a bulk -a using > ALLOW_MAKE_JOBS=3Dyes along with allowing 32 builders. > This so 32*32 or so potentially for load average(s) > at times. There is 128 GiBytes of RAM and: >=20 > # swapinfo > Device 1K-blocks Used Avail Capacity > /dev/gpt/OptBswp480 503316480 0 503316480 0% >=20 > I'm not so sure that such a high load average bulk -a > is reasonable for a debug kernel build: unsure of > resource usage for such and if everything could be > tracked as needed. So I'm testing a non-debug build > for now. >=20 > I have built the kernels (nodbg and dbg), installed > the nodbg kernel, rebooted, and started: >=20 > # poudriere bulk -jmain-amd64-bulk_a -a > . . . > [00:01:22] Building 34042 packages using up to 32 builders > . . . >=20 > The ports tree is from back in mid-July. >=20 > I have a patched up top that records and reports > various MaxObs???? figures (Maximum Observed). It > was recetnly reporting: >=20 > . . .; load averages: 119.56, 106.79, 71.54 MaxObs: 184.08, 112.10, = 71.54 > 1459 threads: . . ., 273 MaxObsRunning > . . . > Mem: . . ., 61066Mi MaxObsActive, 10277Mi MaxObsWired, 71371Mi = MaxObs(Act+Wir+Lndry) > . . . > Swap: . . ., 61094Mi MaxObs(Act+Lndry+SwapUsed), 71371Mi = MaxObs(Act+Wir+Lndry+SwapUsed) Status report at about 1 hr in: [main-amd64-bulk_a-default] [2023-08-19_11h04m26s] [parallel_build:] = Queued: 34435 Built: 1929 Failed: 9 Skipped: 2569 Ignored: 358 = Fetched: 0 Tobuild: 29570 Time: 00:59:59 Not hung up yet. =46rom about 10 minutes after that: . . . load averages: 205.56, 181.58, 153.68 MaxObs: 213.78, 182.26, = 153.68 1704 threads: . . ., 311 MaxObsRunning . . . Mem: . . ., 100250Mi MaxObsActive, 16857Mi MaxObsWired, 124879Mi = MaxObs(Act+Wir+Lndry) . . . Swap: . . . 5994Mi MaxObsUsed, 116589Mi MaxObs(Act+Lndry+SwapUsed), = 127354Mi MaxObs(Act+Wir+Lndry+SwapUsed) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Aug 19 20:28:37 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSr152Zzlz4mRjJ for ; Sat, 19 Aug 2023 20:28:41 +0000 (UTC) (envelope-from ronald@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSr146wq9z3H4d; Sat, 19 Aug 2023 20:28:40 +0000 (UTC) (envelope-from ronald@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692476921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZlxCfJ9hivPThkndFV6sb864zYXAnvBaQnV5BQ1oxQ4=; b=yivB+U0irsY8v7SaSzgXjaRZpiM37KygIwDM/hPEzKU+qcXVTQoT33UN/fXuyTOglnuCCX xqvIUQWOc9VBWKFiTeBkSSsg/jQrifEwBd/RpMAT0kWnvq7X2fdBohobVuB5mwBdMRyfhP Jx2e47UlOx4UcspfSfV1wDDPmoZiVD6kJF3tjL82kecWCjF0/ctXvWKDHY52Y3xfXsaqat JhWXAAGLNcOtHi69Gdyu0QjrGkQjRn1UQJuVuJE6SM8bRQL1vf+wH1uo/zk7p/rlAlukIy twrKjE8QoRipUJDWY6ToXY7vCh5D4Y8AYqMPeUAdbtaTI8S4109UoFXU61klHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1692476921; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZlxCfJ9hivPThkndFV6sb864zYXAnvBaQnV5BQ1oxQ4=; b=AMitsX/QIKT1YHpEjjZ/QpRLqE70GNUKLu4jK8RdkgexgutrnrvyK7zxvUves3qjMZ2zzz KcOO7AmP/ofmKL12CQ/7bnz43sBnxO3sVu1TCQyXT96oDpFe1F1svTV69C4PNm9uPpt1Lc LrE7KPh5/omGUVebh1M1LIYvEXR34pTNyMtLtqb4TyVxJmse5ifQIYTm7W7dJVWMW/u4Z1 IgTqBosBIxvKEjh64qnF8+YRalc6fZJzqKLfHu1VFJ8TX0QWOCdAsHXGdCmjpylvcVr2bs sDqWvo5AFFAwrCUJ8zqe2ZQtUMytS0TXv0a1gAmHIR2xsCDnLW2cc7+uiBmZKQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1692476921; a=rsa-sha256; cv=none; b=q1V8iMeT6MwaKbLwWwO/4YK3txvcRQtXyhTjOzQgRRxe74QjnNMLY/kRgZNaCwHHnbfH+h yEFrfUfi78IJy4A97SJ1RN3RM69T/J7sSlVl6ajiXU1R2wyzUMrueOut7KIjyrXhe6a7ay LjMFEsISrCVDlc+yq2jUIzf/n8FXI1V4cGM4uPkDgcd9MaZR7I17VqTLL8ehgy1ziCDr7W V+QyhnlRRAM34+DR7HW5xXoKie6uBao+As7zPDMVQKuXhhzL7mCHtVMDQr2oIp6u7u4Q46 y8CxArMLClCY6u5GkqD5Z8U0oeD1ilUbG/cO8l7QEM8ceg0WaNeKkGre91AnvQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from [192.168.1.109] (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: ronald/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4RSr143ZMBzpB1; Sat, 19 Aug 2023 20:28:40 +0000 (UTC) (envelope-from ronald@FreeBSD.org) Message-ID: Date: Sat, 19 Aug 2023 22:28:37 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.14.0 Subject: Re: building with llvm16 pkg fails in tests Content-Language: en-US To: Brooks Davis Cc: freebsd-current@freebsd.org References: <76ecade1-3f42-a314-825d-6f1ba0b2dcaa@FreeBSD.org> From: Ronald Klop In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 T24gOC8xNy8yMyAyMTozMywgQnJvb2tzIERhdmlzIHdyb3RlOg0KPiBPbiBUaHUsIEF1ZyAx NywgMjAyMyBhdCAxMjo0NTowNlBNICswMjAwLCBSb25hbGQgS2xvcCB3cm90ZToNCj4+IEhp LA0KPj4NCj4+IFRvIHNhdmUgdGltZSBvbiBteSBSYXNwYmVycnkgUGkgSSB3b3VsZCBsaWtl IHRvIGJ1aWxkIEZyZWVCU0QgdXNpbmcgYSBsbHZtIHBrZyBpbnN0ZWFkIG9mIGxsdm0gaW4g dGhlIHRyZWUuDQo+Pg0KPj4gTXkgL2V0Yy9tYWtlLmNvbmY6DQo+PiBXSVRIT1VUX1RPT0xD SEFJTj15ZXMNCj4+IExEPS91c3IvbG9jYWwvbGx2bTE2L2Jpbi9sZC5sbGQNCj4+IENDPS91 c3IvbG9jYWwvbGx2bTE2L2Jpbi9jbGFuZw0KPj4gQ1hYPS91c3IvbG9jYWwvbGx2bTE2L2Jp bi9jbGFuZysrDQo+PiBDUFA9L3Vzci9sb2NhbC9sbHZtMTYvYmluL2NsYW5nLWNwcA0KPj4g T0JKQ09QWT0vdXNyL2xvY2FsL2xsdm0xNi9iaW4vbGx2bS1vYmpjb3B5DQo+Pg0KPj4gI1dJ VEhPVVRfQ0xFQU49eWVzDQo+Pg0KPj4NCj4+IFRoaXMgZmFpbHMgaW46DQo+Pg0KPj4gL3Vz ci9sb2NhbC9sbHZtMTYvYmluL2NsYW5nKysgLU8yIC1waXBlIC1mbm8tY29tbW9uIC1mUElF IC1Xbm8tZm9ybWF0LXplcm8tbGVuZ3RoIC1ub2J1aWx0aW5pbmMgLWlkaXJhZnRlciAvdXNy L2xvY2FsL2xsdm0xNi9saWIvY2xhbmcvMTYvaW5jbHVkZSAtZnN0YWNrLXByb3RlY3Rvci1z dHJvbmcgLVdzeXN0ZW0taGVhZGVycyAtV2Vycm9yIC1XYWxsIC1Xbm8tZm9ybWF0LXkyayAt VyAtV25vLXVudXNlZC1wYXJhbWV0ZXIgLVdwb2ludGVyLWFyaXRoIC1XcmV0dXJuLXR5cGUg LVdjYXN0LXF1YWwgLVd3cml0ZS1zdHJpbmdzIC1Xc3dpdGNoIC1Xc2hhZG93IC1XdW51c2Vk LXBhcmFtZXRlciAtV2Nhc3QtYWxpZ24gLVdjaGFyLXN1YnNjcmlwdHMgLVdkYXRlLXRpbWUg LVdtaXNzaW5nLXZhcmlhYmxlLWRlY2xhcmF0aW9ucyAtV25vLWVtcHR5LWJvZHkgLVduby1z dHJpbmctcGx1cy1pbnQgLVduby11bnVzZWQtY29uc3QtdmFyaWFibGUgLVduby1lcnJvcj11 bnVzZWQtYnV0LXNldC1wYXJhbWV0ZXIgLU8wIC1nMCAtUXVudXNlZC1hcmd1bWVudHMgLUkv dXNyL29iai91c3Ivc3JjL2FtZDY0LmFtZDY0L3RtcC91c3IvaW5jbHVkZS9wcml2YXRlIC1J L3Vzci9zcmMvY29udHJpYi9nb29nbGV0ZXN0L2dvb2dsZW1vY2svaW5jbHVkZSAtSS91c3Iv c3JjL2NvbnRyaWIvZ29vZ2xldGVzdC9nb29nbGVtb2NrIC1JL3Vzci9zcmMvY29udHJpYi9n b29nbGV0ZXN0L2dvb2dsZXRlc3QvaW5jbHVkZSAtSS91c3Ivc3JjL2NvbnRyaWIvZ29vZ2xl dGVzdC9nb29nbGV0ZXN0IC1JL3Vzci9vYmovdXNyL3NyYy9hbWQ2NC5hbWQ2NC90bXAvdXNy L2luY2x1ZGUvcHJpdmF0ZSAtREdURVNUX0hBU19QT1NJWF9SRT0xIC1ER1RFU1RfSEFTX1BU SFJFQUQ9MSAtREdURVNUX0hBU19TVFJFQU1fUkVESVJFQ1RJT049MSAtZnJ0dGkgLVduby1k ZXByZWNhdGVkLWNvcHkgLVduby1zaWduZWQtdW5zaWduZWQtd2NoYXIgLURHVEVTVF9IQVNf UE9TSVhfUkU9MSAtREdURVNUX0hBU19QVEhSRUFEPTEgLURHVEVTVF9IQVNfU1RSRUFNX1JF RElSRUNUSU9OPTEgLWZydHRpIC1Xbm8tZGVwcmVjYXRlZC1jb3B5IC1Xbm8tc2lnbmVkLXVu c2lnbmVkLXdjaGFyIC1mUElFIC1zdGQ9YysrMTQgLVduby1kZXByZWNhdGVkLWNvcHkgLVdu by1lcnJvcj1pbmNvbnNpc3RlbnQtbWlzc2luZy1vdmVycmlkZSAtV25vLWVycm9yPW1pc3Np bmctdmFyaWFibGUtZGVjbGFyYXRpb25zIC1Xbm8tZXJyb3I9c2lnbi1jb21wYXJlIC1Xbm8t ZXJyb3I9dW51c2VkLXBhcmFtZXRlciAtV25vLWMrKzExLWV4dGVuc2lvbnMgIC1XbCwtenJl bHJvIC1waWUgIC0tbGQtcGF0aD0vdXNyL2xvY2FsL2xsdm0xNi9iaW4vbGQubGxkIC1vIGdt b2NrLWFjdGlvbnNfdGVzdCAgZ21vY2stYWN0aW9uc190ZXN0Lm8gLWxwcml2YXRlZ21vY2tf bWFpbiAtbHByaXZhdGVnbW9jayAtbHByaXZhdGVndGVzdA0KPj4gbGQubGxkOiBlcnJvcjog dW5kZWZpbmVkIHN5bWJvbDogdGVzdGluZzo6aW50ZXJuYWw6OkRlYXRoVGVzdDo6Q3JlYXRl KGNoYXIgY29uc3QqLCB0ZXN0aW5nOjpNYXRjaGVyPHN0ZDo6X18xOjpiYXNpY19zdHJpbmc8 Y2hhciwgc3RkOjpfXzE6OmNoYXJfdHJhaXRzPGNoYXI+LCBzdGQ6Ol9fMTo6YWxsb2NhdG9y PGNoYXI+PiBjb25zdCY+LCBjaGFyIGNvbnN0KiwgaW50LCB0ZXN0aW5nOjppbnRlcm5hbDo6 RGVhdGhUZXN0KiopDQo+Pj4+PiByZWZlcmVuY2VkIGJ5IGdtb2NrLWFjdGlvbnNfdGVzdC5j Yw0KPj4+Pj4gICAgICAgICAgICAgICAgZ21vY2stYWN0aW9uc190ZXN0Lm86KHRlc3Rpbmc6 Oihhbm9ueW1vdXMgbmFtZXNwYWNlKTo6QnVpbHRJbkRlZmF1bHRWYWx1ZURlYXRoVGVzdF9J c1VuZGVmaW5lZEZvclJlZmVyZW5jZXNfVGVzdDo6VGVzdEJvZHkoKSkNCj4+Pj4+IHJlZmVy ZW5jZWQgYnkgZ21vY2stYWN0aW9uc190ZXN0LmNjDQo+Pj4+PiAgICAgICAgICAgICAgICBn bW9jay1hY3Rpb25zX3Rlc3QubzoodGVzdGluZzo6KGFub255bW91cyBuYW1lc3BhY2UpOjpC dWlsdEluRGVmYXVsdFZhbHVlRGVhdGhUZXN0X0lzVW5kZWZpbmVkRm9yUmVmZXJlbmNlc19U ZXN0OjpUZXN0Qm9keSgpKQ0KPj4+Pj4gcmVmZXJlbmNlZCBieSBnbW9jay1hY3Rpb25zX3Rl c3QuY2MNCj4+Pj4+ICAgICAgICAgICAgICAgIGdtb2NrLWFjdGlvbnNfdGVzdC5vOih0ZXN0 aW5nOjooYW5vbnltb3VzIG5hbWVzcGFjZSk6OkJ1aWx0SW5EZWZhdWx0VmFsdWVEZWF0aFRl c3RfSXNVbmRlZmluZWRGb3JOb25EZWZhdWx0Q29uc3RydWN0aWJsZVR5cGVfVGVzdDo6VGVz dEJvZHkoKSkNCj4+Pj4+IHJlZmVyZW5jZWQgNCBtb3JlIHRpbWVzDQo+Pg0KPj4gbGQubGxk OiBlcnJvcjogdW5kZWZpbmVkIHN5bWJvbDogdGVzdGluZzo6RXhwZWN0YXRpb246OkV4cGVj dGF0aW9uKHN0ZDo6X18xOjpzaGFyZWRfcHRyPHRlc3Rpbmc6OmludGVybmFsOjpFeHBlY3Rh dGlvbkJhc2U+IGNvbnN0JikNCj4+DQo+Pg0KPj4gQW55IHRob3VnaHRzIG9uIGhvdyB0byBm aXggdGhpcz8NCj4+IENvbXBpbGluZyB3aXRoIHRoZSBpbiB0cmVlIGxsdm0gZG9lcyB3b3Jr IHByb3Blcmx5Lg0KPiANCj4gRGlkIGl0IHdvcmsgd2l0aCBleGFjdGx5IHRoaXMgZ2l0IHJl dmlzaW9uPyAgSSBzdXNwZWN0IGFuIGlzc3VlIHdpdGggdGhlDQo+IHJlY2VudCBnb29nbGUg dGVzdCB1cGRhdGUgcmF0aGVyIHRoYW4gYW4gbGx2bTE2IGlzc3VlLiAgTm90ZSB0aGF0IGZv cg0KPiBldmVyeSBzeW5jIHRvIGdpdGh1YiB3ZSBidWlsZCB0aGUgdHJlZSB3aXRoIHRoZSBs bHZtMTYgcG9ydCAoYWxsIGJlIGl0DQo+IG9uIGFtZDY0IGJ5IGRlZmF1bHQpLg0KPiANCj4g SXQncyB3b3J0aCBub3Rpbmcgb25lIGRpZmZlcmVuY2UgYmV0d2VlbiB5b3VyIGNvbmZpZ3Vy YXRpb24gYW5kIHRoZQ0KPiBDSSBvbmU6ICBXZSBkb24ndCBzZXQgQ0MgYW5kIGZyaWVuZHMg ZGlyZWN0bHkuICBJbnN0ZWFkIHdlIHVzZQ0KPiBDUk9TU19UT09MQ0hBSU49bGx2bTE2Lg0K PiANCj4gLS0gQnJvb2tzDQoNCg0KSGksDQoNCldoYXQgSSB3b3VsZCBsaWtlIHRvIGFjY29t cGxpc2ggaXMgdGhpczoNCg0KQ1JPU1NfVE9PTENIQUlOPWxsdm0xNg0KV0lUSE9VVF9UT09M Q0hBSU49eWVzDQoNCnllcyB8IG1ha2UgZGVsZXRlLW9sZCBkZWxldGUtb2xkLWxpYnMNCm1h a2UgYnVpbGR3b3JsZCBidWlsZGtlcm5lbA0KDQpTbyBJIGNhbiBydW4gYSBzeXN0ZW0gd2l0 aCBvbmx5IGV4dGVybmFsIHRvb2xjaGFpbi4NCkJ1dCBkb2luZyB0aGlzIHF1aWNrbHkgZXJy b3JzIG91dCBiZWNhdXNlIC91c3IvYmluL2NjIGRvZXMgbm90IGV4aXN0IGFuZCB0aGUgYnVp bGQgdXNlcyB0aGF0IGV2ZW4gdGhvdWdoIENST1NTX1RPT0xDSEFJTiBpcyBzZXQuIFRvIGNp cmN1bXZlbnQgdGhhdCBJIHNldCBDQywgZXRjLiBpbnN0ZWFkIG9mIENST1NTX1RPT0xDSEFJ Ti4NCg0KUmVnYXJkcywNClJvbmFsZC4NCg0K From nobody Sat Aug 19 20:41:56 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSrJh1TMSz4mSt3 for ; Sat, 19 Aug 2023 20:42:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSrJg0Q9cz3Jwg for ; Sat, 19 Aug 2023 20:42:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=Man5feW7; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692477729; bh=DbmaRmo6j39C+1MCwztFu9UUrPw4Q/QoV87Xnv4By38=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=Man5feW7Ab4cWO0EVU3/a5f9iv9uNqzj+SIm8j7tx34VVucq7jDZX0bNe/v9j0DhRRIvMy+1+sQFlBD/u8gKjMw4k420fk34f+NelSggMU7KR6TEFntk5Vaxk3gT/m4NtDZhV+9NvTtpZ/3kqKREY/v5vZDEXaKWey2t3xg+7CrfIJGngL4YB91f2qmloZc4xMmDWgs+G0AVaWlhZ6LBcKSrCthqF38MfZ+DPj7OADp7uP7q0VvUVhbRNHUOQ2or3UOp+PIuaPFM2KEihY35jbJ79MLtD0nC29GIporLA0ALO1EQzH7TwovIej3s5p3p+XLDBRpUJxwUaf4qJVUdRw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692477729; bh=VX6DL9T5W8RGe/2MLth9oKKf+BPPbSjoOsn2571x3VT=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=T5oQGwTWn7U1qKaL5eVVNhxOVz81QVlsOrV9AQHFmjjL8D/BZg8KnEu2hmnfv59Uwhf4cAFlluq7+JsDv4JlLJp3X6cQxYwumu65Utg9FlLoEpgvBtQ1lq7PK3kyeSxOl6TfBZxqW5v/KjRJnHsGeMWg+l8sNbdOatGg2xoKCLOGOgr/NuZYrkWbrZ0HaSneEMi9AvyOCP3s/1rpuT1mEeGT8dbvRAULVHe6njDaArZBu9s9dZNqTjofa4Kt5fiLD8Vp/togMAoik73JGnPYfGkW+dhz15I9Pdwq3LmjkfgI7YJeU0Rg0WvcDEX2FEAuUH4zwnBwazlkUgOqCQincg== X-YMail-OSG: CEdfHGYVM1mkajN_wjDN4nYkvS6pSitVDGrHq2F1gNC2qEI7hHbJb_xH9yHkMdz .6QvsnTIRI2zLaRe4BtAJtA1E6KKkQ23Ftb9rbibwx1FqXVHbcqOG.JR08Yz1bwHUcQqrxxKjrAU b6748AyQ_JTCpPfC0eIiwBz3INrcq9WuwgEOzGkw5tlj3kwQkcLiANduZk72y9SR9mT5lqXJSGbs 8j6TsstF1vYW7z8Ydwp5MPj_YD4J3cI2pKCUfx5xnKrqGQ7iIUaY8E9vUT9Jjq1COX_XFYL9sNVk Tfch2tJ4XzI4lS18rgQt5K2I0SLxOrUxoehagum1HWgP_i2pIIqn7nN8HURmrjx1oCzlbhvgUs8d 6gBUsSCpTKssO19RrkZl0.Ur20683HUAadAejuUiOYtsC4kjxuUWmw3dJfRB624hy2YhseazDHva PoKamL9CSZphfvomhc_Str8xJYHmVjUHvARq_Gyo8uH4muk4hn39ZZW9XW7bk0EyhsmHbV5HiKAI fyt3x35RJAOzNMRxpPKXXO.YMmDNoM7KsT7hoNSxHl9BHyg1jEYVJNA6nFfZX4ojlwNnsNIfpdv6 zlZ5Oojx9Blw75hlYPMcN419qUvrESgcLwGuOie3p1m8dg.Jf6cEP45Dvkt.2DVjVPtCQhYd5rsN eQRZQvNDb9Lj1V19LgwUz3HqDbnT_5mDVxoZhFBxjdH.iMwP4oZE1hD8cpvPfg_T6CEpBxCX5cFR pZgJ9keO6GWqhbEnszCviEgeqaE6nWPXhxiebr7kF0aqFebVzJkWvTXnkiQO3xkllUyHM5QHOfUn zV3LSIAbDMoYP6YkYw_IPsj.uniiqHQnrVIjWMIBo4rQwNMJ1v2JRGMG4ZP3rQRaJa1XJqawEVDI nYdjfjEevwZg9uOGOo8smy2QEtxyvcIGXwypJYQgPa1gYV6Bf2Lx3uD.dpr69eGzPvtifDVuc4I6 KkWS1952oY3V6_WUSHfAWtc10CTNisc_1eSrU_6pEw5JlpdWwVR5fl62wCwcFoOs2zX9PmvM3mc5 F2KBwR82o_YmI_O4iJ3yu_D7IS.m.VAf6Lpw4276vxazseCUEpavPhJ7QlvZWF_1tbMMoRaQ3g.v vqf6X_GUQEq85KqPaAhVxEk9LhV3.2inTqm1N23Q0QvIVLgnHT9xb3UVPBhUNC2DM71_swk6qV9a SB9QYDUCMMN.c5H094Ig3gTvqmYUBhH9VRm2D9FQ5bYvUKXCNHfmI9zoJVd.FI4oUOpWG6wkVa82 NUIXTPzcgbUgL6BURU1VEQOVeNpWhbDyJOXap26KW9TaRR7S7JMSFqbX0X.WpSLkInqfWh5JZh3D 51fNFMH5DJWUhl4kFr7Y2zJ7sjT0A_LrD8BUg32YUfOgQ2EUjX9pp.bXgia5dK_YkT6K89G_qPKf cMQChyUe4AidM2l9er5DYf1n8q67W_sx9sWPzLlw0aGi2xnuS3BL3G9RH1t3K1cZvIX9Y3DWb43n 47YmeXeKzqyW9K_hWWy.8Dq2rhDTFtWh5SNXFW7CGqeiVtDU01Pk9HyLIV9kXrnvXmb0N1Pk6iqB apfDcrU9LhkP_IaSKOLrDNyVS18drrp4i9gXXO90MFQWmOKCs_s2HzJBZD3hHBouQoz1jLUAoYuo x6fak94EGo88WHl1UhbEfmTCgskscb2jvxef5qFLBq5kkM5jYouVTSpaVhbYk6AbQSS1iUzXJ96I mf.P8hKaIbFaR8YccI8h03Q3i3JH2fTFxTQjYSCwGHfYRQQZNBBOEQ3IYodoRaV3syQe6VRRd6Dp _tpxiRegZTbuNQc.pTD1swR_203bT444hu4CNZZBzMEIVdQD5lS99JQ_10MTjqI.Iayi68LMHf3U fmOJddSMKQD3bJzNzyMQrugFBFFL6B4VZ.jxDVT2VcwQqUc.icnDta1_W_yJzDOxZJkJZfH5_qzg A0ANEoSzdoV1voU8r6oicxsYdiTtcfo5gTNpFDbjtWbZxT27RjjV1spdJbc1Pcxr.rbuWF9FirB0 w_1DQXZ3Tg0ymuxjXJbVdDceCP1sIeO11DZk2JpMTcypSJ2AiL2zIu0XoC86ilNXKIKHUNCgmqWF fdbnVHm9TaXmII13jUGukb7BcYWgG51v8uOQ3rfnzK.ul8vyEMShzEk3fSJq.yeDCAnIVxUajNEI uwtQUCBPgkNaDEAdFKyxo4LESLDBlr_NOqBeYkXHva8DEcl9cTRpzM2dBnhZA5EuLFgRjFVpXgRn h1K1j9hxIARTS4RO1jB07R7zf2MTQW_IWLMQ6NVcDVsniDdLA8Q6zdjrsrg0C X-Sonic-MF: X-Sonic-ID: fb761505-5ba6-44a6-b11c-3c2dc42406e2 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Aug 2023 20:42:09 +0000 Received: by hermes--production-gq1-6b7c87dcf5-j6k2s (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 369796f3a1c3d0fd00d8a854784b184f; Sat, 19 Aug 2023 20:42:07 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14 Date: Sat, 19 Aug 2023 13:41:56 -0700 References: <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> <0F2C42B4-36FF-443A-A174-5B0CC57C4FC7@yahoo.com> <3AA253E3-C4F0-4AA3-9C37-D77E7527A458@yahoo.com> To: Current FreeBSD In-Reply-To: <3AA253E3-C4F0-4AA3-9C37-D77E7527A458@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.83:from]; RCPT_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.83:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSrJg0Q9cz3Jwg [I forgot to adjust USE_TMPFS for the purpose of the test. So I'll later be starting over.] On Aug 19, 2023, at 12:18, Mark Millard wrote: > On Aug 19, 2023, at 11:40, Mark Millard wrote: >=20 >> We will see how long the following high load average bulk -a >> configuration survives a build attempt, using a non-debug kernel >> for this test. >>=20 >> I've applied: >>=20 >> # fetch -o- https://github.com/openzfs/zfs/pull/15107.patch | git -C = /usr/main-src/ am --dir=3Dsys/contrib/openzfs >> - 13 kB 900 = kBps 00s >> Applying: Remove fastwrite mechanism. >>=20 >> # fetch -o- https://github.com/openzfs/zfs/pull/15122.patch | git -C = /usr/main-src/ am --dir=3Dsys/contrib/openzfs >> - 45 kB 1488 = kBps 00s >> Applying: ZIL: Second attempt to reduce scope of zl_issuer_lock. >>=20 >> on a ThreadRipper 1950X (32 hardware threads) that is at >> main 6b405053c997: >>=20 >> Thu, 10 Aug 2023 >> . . . >> =E2=80=A2 git: cd25b0f740f8 - main - zfs: cherry-pick fix from = openzfs Martin Matuska=20 >> =E2=80=A2 git: 28d2e3b5dedf - main - zfs: cherry-pick fix from = openzfs Martin Matuska >> . . . >> =E2=80=A2 git: 6b405053c997 - main - OpenSSL: clean up botched = merges in OpenSSL 3.0.9 import Jung-uk Kim >>=20 >> So it is based on starting with the 2 cherry-pick's as >> well. >>=20 >> The ThreadRipper 1950X boots from a bectl BE and >> that zfs media is all that is in use here. >>=20 >> I've setting up to test starting a bulk -a using >> ALLOW_MAKE_JOBS=3Dyes along with allowing 32 builders. >> This so 32*32 or so potentially for load average(s) >> at times. There is 128 GiBytes of RAM and: >>=20 >> # swapinfo >> Device 1K-blocks Used Avail Capacity >> /dev/gpt/OptBswp480 503316480 0 503316480 0% >>=20 >> I'm not so sure that such a high load average bulk -a >> is reasonable for a debug kernel build: unsure of >> resource usage for such and if everything could be >> tracked as needed. So I'm testing a non-debug build >> for now. >>=20 >> I have built the kernels (nodbg and dbg), installed >> the nodbg kernel, rebooted, and started: >>=20 >> # poudriere bulk -jmain-amd64-bulk_a -a >> . . . >> [00:01:22] Building 34042 packages using up to 32 builders >> . . . >>=20 >> The ports tree is from back in mid-July. >>=20 >> I have a patched up top that records and reports >> various MaxObs???? figures (Maximum Observed). It >> was recetnly reporting: >>=20 >> . . .; load averages: 119.56, 106.79, 71.54 MaxObs: 184.08, 112.10, = 71.54 >> 1459 threads: . . ., 273 MaxObsRunning >> . . . >> Mem: . . ., 61066Mi MaxObsActive, 10277Mi MaxObsWired, 71371Mi = MaxObs(Act+Wir+Lndry) >> . . . >> Swap: . . ., 61094Mi MaxObs(Act+Lndry+SwapUsed), 71371Mi = MaxObs(Act+Wir+Lndry+SwapUsed) >=20 > Status report at about 1 hr in: >=20 > [main-amd64-bulk_a-default] [2023-08-19_11h04m26s] [parallel_build:] = Queued: 34435 Built: 1929 Failed: 9 Skipped: 2569 Ignored: 358 = Fetched: 0 Tobuild: 29570 Time: 00:59:59 >=20 > Not hung up yet. >=20 > =46rom about 10 minutes after that: >=20 > . . . load averages: 205.56, 181.58, 153.68 MaxObs: 213.78, 182.26, = 153.68 > 1704 threads: . . ., 311 MaxObsRunning > . . . > Mem: . . ., 100250Mi MaxObsActive, 16857Mi MaxObsWired, 124879Mi = MaxObs(Act+Wir+Lndry) > . . . > Swap: . . . 5994Mi MaxObsUsed, 116589Mi MaxObs(Act+Lndry+SwapUsed), = 127354Mi MaxObs(Act+Wir+Lndry+SwapUsed) Just relized that I'd forgotten to reconfigure the USE_TMPFS=3Dall to be USE_TMPFS=3Dno so what I've done so far is not a great test. I'll still probably let it reach 3hr and get the summary information before I stop it, adjust USE_TMPFS, and start over from scratch. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Aug 19 22:41:04 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RStyC6fWGz4qK4j for ; Sat, 19 Aug 2023 22:41:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RStyB5YvMz3Spj for ; Sat, 19 Aug 2023 22:41:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="VBh3/DFG"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692484880; bh=rmnu/zQViM1luMQlR/MUpxLw72I+KTFc5/eKSVA8wwo=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=VBh3/DFGT5IHVuOVMjAk4Q3h6osfMRKVHBFfAuf+rjY2jjGEJL3PozMnH4WdLC0ZktZ6KBsznohrfc7EuuvpWTqUh03Ay5zTYNGmmfVse1mo6S4W4sJAIFWP0MTXxcOeRoEV09ygJqLKGkTKmP87TYGxgmQhWMo2Z2linu9nT5APU7Uog2bE9475TPyjE6BITnR9hYWpgw0DrZ/r/PQIjWNEOn8NpyhwymMIZM/jYxOnxJ7Ytfx8HjRMIIaFBjpYqBRHVXoLVMbdamZKPdVaNQiFyanATFivYp3ChaU/1U78tjT3KHSb/XDefS6wTsbGP3Br6/QrqkvQksT9eNbcOw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692484880; bh=AfZy4r/SkCtPKum5giswd3ueagdCSBJ1Z9Ovgdcm72F=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Bzv2yFZuiQbZ/2xfmeSgGzjqS/skDfVkAi4E89rYr0+PY2klEYRRsA0H/pa0Q8APaBXZ079f5UQ9MppsRsSGgRuKcvZamiRHTA6xseIS1r3/IeldtOmha5+ydV9nxHRctSf1vS3RibjvG8WNlSOgD70kmVrdUa8FCea9z4rUctai86VNkn+JNXLRIJuVaxVScP5xpk+SGQ9V1E89efU+9w7AF3pLex5StTx/tZAznqKbOdNkf2Aonhd618BzZvon9BM92LkiiSQk4qn5Gri6fstzCEdpuC+kct1xlLpzBAhhNJ3KaEuAUFNHZ8QmGXnonGv5Uc6kn5itVlhWF6dLqA== X-YMail-OSG: t9TvUH0VM1ngOhDDGBgu.3bPw0ljH6K2F1_J4rvN0rmVNZU44b5DSkP_g_EfOl_ ZYRRiPnIwvuRBIWCq99iUbksHaxlOFk6nfJO7aFhNM35XjPKZEcB9IFQ9e1EnaTK2xLxoFuTw15n 9QHhkmCSNJKnkXkVElwtxnd0HQTYXgbRtenD5bYcMTs5or9sdBQF9wmlhIoFbxB3_ntO.si29BfB DlFeyQWzKRhNOrP_aRxTc_2_gKLs32VHwtZmUehU7_NHIKWJJinNC554O4DRx26HBxCu2qx8AGj. bzQI65JN9BKHlJw1UGrawEw.BjYb9fMhQLUHF15SURx_cVHVlv1FRrT6jkj_XzukAOdmzU1vrsQS JXMDFgZZqToaQGjzZQmPjMvEX5923rGu28VSCtMGbXq.PxOI3RGIZpVjFw8L9dChdU_elzQlUXpO 5_bPV2lrCdhYE3o3KnhDVa0VGGfJHzm4yYJbXZw415ugqFBDt71gAQIzXC9dSHXbp1KXRrBJuABx SsXWabrVAzA650HfqpnxmBAUpX9JFLowzVop_lNNEBLUPpxRjwPhjKyoX5dtIt0PyhM6Tt_w1LNz I56qtk7s8ZE07oBTHD4zlk4R.CYuOZp77sYz5tRXoRp_qsJoje31QcYaMW7Ikna4IpvW6gb9HLLR RoMheIisELGGTe8XqXRqaD5H9aAxoon5KeCurpvQlpx4rbzn4ag5_OXAS8tRHmleYcIV0ZOpe9eW 9n7Z90vk8pLO_Mflpa7iJ7avqGGrWFlmDynfazPCWU6.hklgSts3NGiQTEScUEPmJSG9KzMlyNyv ZwX7Z0Zjbvzgd6s_c91FA9X8pI4E4I42yH5TRjEFYl8DjAGG78ADKCaGN2Kjc0XYjk1OeY8UTpP6 vxLaLmgB9rGXpzj4NVaFfov7YEtRUrFN582pivNRWls04yLUp0zI3uTJcRfWBtQPJEq5I.4IrRkD wRajTJ4Eh6deYqxWFdfA3NKLXnZl9vDxTtieDj8CqM_1nbfR7V1vRlMzCovU6SfkcuR5Vx.p7LOv wC.kfjKopaDbkTdNposq_qboAg9XaONt7qjv1Dy9TDzjaVgWTyVXEUAMmTOcQvJ0F.W3_neS5S_P hmHmHW9vJG_eA66nkL97yS3efN.sAe5n2mhcKBdS3qdW2YC6JGESbPOxP63op25ivX.SrX5SinIG fpmRJc0DUhPYi7Q7dclPSf2QPgJZLhOGfFNkezKMNbGawcBj2BAe5KZ4vUlPhuGDOkS.DQ.VfJR3 YshcXGf.WK6mIOJxaZ4OLW5Cr9XWkThKgSceaUO_QXOzmQb6hrFTomoR7CCso2LfretqT2nd2KTZ Y3eBRxRaEz30cU6q6hmYmoQEbTvQtXvyQBJbCZ1OcnjVpHI0Qjh0rVrovSnrgp0Hn6KSgg24j7Zw AYqz6O4pGfsT1lKDP5BOHa.rpAKP6YdBfqyP7cIrA8JHNoCWWB.rJCGlYhg77mZcawVd079QYkpu A6bUEDfu64MkMWsLU4WztrF5DUFtV4rvcxenm0ApAGd4SaXik4UjnRcJhm3wmQ5PZV1eyvQI5Lop zQl5F6t5GOz8xi.XsrirGbht3SKGEdNtXs1i8tbVsTiXtu1ro2MkfQv8AKsXXM006aza9bIww34a xHuj_Aq9VMOv2Osdb95x2HmqCHtLIgLx6BFo4nXaIAHXtCwCw2sTXqWD9..3c6bBwkyb6iqMcLhm GivjEBS_XorZZugvr1WcHGtPOVhDceI5RhIvI2e_vKFiXSCNTFN6jl3aD1_dV49aioVzoG1XN.iA exOdmwMDsTy2gAbKKzZdoSfdN22zXYGP.7WZ_.C5_va3X31RpL1zg.iS7vBawxXxLZxYBPLBafRS KV9YVv3IssmqMhGgksG7ciIsCQ4mJgUtJazbjkDa0AgJycFEQofMTRKGbl.SQMDF4YgcqqjKJJ8y FQbZS77b0pH6OR5HqzaNXJhRwwgowPZ7lKFBUAtiisbsmA2LYaE_8b0Hs3hmWttMVmXIKn8E75g7 s1KJracluXYRXm5uZds6DQ_NttD.VW8OV6RxD.VKpT8zr6m.f1bBGFfcyPYysl.AsTKprggSbogV BtoRPb9J4rIxGXpJimTIL9FmfunC6xLauhn2x98myFAgU_cUkt_GtboMVr3SQBkj4kTbQ6NyRtyn 5AdifR9.4W10nNSfVLsl9ULxnab_44FIVh_leez45.hhGv5qk2kIMSsHe7d7cVJwTnRa7OudSwC6 qPROTLneXLjY9TDuuTd2RPs7h5ULLrduAlC9WgzXUvsm1oE6qxkAsaVVkvhITWw-- X-Sonic-MF: X-Sonic-ID: e4c11f37-bb74-49b1-8b52-9844073b8b85 Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Aug 2023 22:41:20 +0000 Received: by hermes--production-bf1-865889d799-r6v2w (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 91cdd64db58aad3a703eaa54ba53c5b0; Sat, 19 Aug 2023 22:41:16 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: ZFS deadlock in 14 [USE_TMPFS=no poudriere messed up from the start, lots of "vlruwk"] Date: Sat, 19 Aug 2023 15:41:04 -0700 References: <59FCB309-4A55-4924-98C4-7ACCA70FD299@yahoo.com> <0F2C42B4-36FF-443A-A174-5B0CC57C4FC7@yahoo.com> <3AA253E3-C4F0-4AA3-9C37-D77E7527A458@yahoo.com> To: Current FreeBSD In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; BLOCKLISTDE_FAIL(0.00)[98.137.69.205:server fail]; RCPT_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.205:from]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.205:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RStyB5YvMz3Spj On Aug 19, 2023, at 13:41, Mark Millard wrote: > [I forgot to adjust USE_TMPFS for the purpose of the test. > So I'll later be starting over.] >=20 > . . . I finally got around to starting a from-scratch bulk -a again (based on USE_TMPFS=3Dno this time). This is with 15107.patch and 15122.patch applied. This is a non-debug kernel experiment. Interstingly it got: [00:01:34] [01] [00:00:00] Builder starting [00:01:57] [01] [00:00:23] Builder started [00:01:57] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.20.4 [00:03:09] [01] [00:01:12] Finished ports-mgmt/pkg | pkg-1.20.4: Success [00:03:21] [01] [00:00:00] Building print/indexinfo | indexinfo-0.3.1 [00:03:21] [02] [00:00:00] Builder starting [00:03:21] [03] [00:00:00] Builder starting [00:03:21] [04] [00:00:00] Builder starting [00:03:21] [05] [00:00:00] Builder starting [00:03:21] [06] [00:00:00] Builder starting [00:03:21] [07] [00:00:00] Builder starting [00:03:22] [08] [00:00:00] Builder starting [00:03:22] [09] [00:00:00] Builder starting [00:03:22] [10] [00:00:00] Builder starting [00:03:22] [11] [00:00:00] Builder starting [00:03:22] [12] [00:00:00] Builder starting [00:03:22] [13] [00:00:00] Builder starting [00:03:22] [14] [00:00:00] Builder starting [00:03:22] [15] [00:00:00] Builder starting [00:03:22] [16] [00:00:00] Builder starting [00:03:22] [17] [00:00:00] Builder starting [00:03:22] [18] [00:00:00] Builder starting [00:03:22] [19] [00:00:00] Builder starting [00:03:22] [20] [00:00:00] Builder starting [00:03:22] [21] [00:00:00] Builder starting [00:03:22] [22] [00:00:00] Builder starting [00:03:22] [23] [00:00:00] Builder starting [00:03:22] [24] [00:00:00] Builder starting [00:03:22] [25] [00:00:00] Builder starting [00:03:22] [26] [00:00:00] Builder starting [00:03:22] [27] [00:00:00] Builder starting [00:03:22] [28] [00:00:00] Builder starting [00:03:22] [29] [00:00:00] Builder starting [00:03:22] [30] [00:00:00] Builder starting [00:03:22] [31] [00:00:00] Builder starting [00:03:22] [32] [00:00:00] Builder starting [00:03:30] [01] [00:00:09] Finished print/indexinfo | indexinfo-0.3.1: = Success [00:03:31] [01] [00:00:00] Building devel/gettext-runtime | = gettext-runtime-0.22 and is still that way minutes later. ^T shows: [00:03:31] [01] [00:00:00] Building devel/gettext-runtime | = gettext-runtime-0.22 load: 13.02 cmd: sh 2187 [vlruwk] 570.19r 0.62u 38.60s 9% 3948k #0 0xffffffff80b7701b at mi_switch+0xbb #1 0xffffffff80bc976f at sleepq_timedwait+0x2f #2 0xffffffff80b76770 at _sleep+0x1d0 #3 0xffffffff80c5b435 at vn_alloc_hard+0x2a5 #4 0xffffffff80c50b72 at getnewvnode_reserve+0x92 #5 0xffffffff829b9b12 at zfs_zget+0x22 #6 0xffffffff829a6a8d at zfs_dirent_lookup+0x16d #7 0xffffffff829a6b5f at zfs_dirlook+0x7f #8 0xffffffff829b6410 at zfs_lookup+0x350 #9 0xffffffff829b182a at zfs_freebsd_cachedlookup+0x6a #10 0xffffffff80c36a0d at vfs_cache_lookup+0xad #11 0xffffffff80c45141 at vfs_lookup+0x581 #12 0xffffffff80c44238 at namei+0x238 #13 0xffffffff80c63b5e at kern_statat+0xee #14 0xffffffff80c64237 at sys_fstatat+0x27 #15 0xffffffff81049a79 at amd64_syscall+0x109 #16 0xffffffff8101f11b at fast_syscall_common+0xf8 [main-amd64-bulk_a-default] [2023-08-19_15h14m10s] [parallel_build:] = Queued: 34435 Built: 2 Failed: 0 Skipped: 35 Ignored: 358 = Fetched: 0 Tobuild: 34040 Time: 00:10:52 ID TOTAL ORIGIN PKGNAME PHASE PHASE = TMPFS CPU% MEM% [01] 00:07:29 devel/gettext-runtime | gettext-runtime-0.22 build = 00:06:32 25.4% 0% [00:11:25] Logs: = /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-08-19_1= 5h14m10s Note the 3:31->11:25 . Top is showing lots of "vlruwk". For example: 362 0 root 40 0 27076Ki 13776Ki CPU19 19 4:23 = 0.00% cpdup -i0 -o ref 32 349 0 root 53 0 27076Ki 13776Ki vlruwk 22 4:20 = 0.01% cpdup -i0 -o ref 31 328 0 root 68 0 27076Ki 13804Ki vlruwk 8 4:30 = 0.01% cpdup -i0 -o ref 30 304 0 root 37 0 27076Ki 13792Ki vlruwk 6 4:18 = 0.01% cpdup -i0 -o ref 29 282 0 root 42 0 33220Ki 13956Ki vlruwk 8 4:33 = 0.01% cpdup -i0 -o ref 28 242 0 root 56 0 27076Ki 13796Ki vlruwk 4 4:28 = 0.00% cpdup -i0 -o ref 27 =20 In other words, it is messed up from the start, not just later. It does suggest that the dbg kernel should not end up with resource problems: not that much gets very far. So I'll probably stop it and substitute the debug kernel, reboot and try again. =3D=3D=3D Mark Millard marklmi at yahoo.com