From nobody Thu Mar 23 01:03:19 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PhnCY1jJyz40JcR for ; Thu, 23 Mar 2023 01:03:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PhnCX0tctz46J3 for ; Thu, 23 Mar 2023 01:03:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=phAh7PiF; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679533413; bh=yTfU9CKUj/eBcDcyOXiYjc3zGn6cMji37M7a3LlJc4w=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=phAh7PiF1jVXX6aTvi8/qk6vSNNNibPzShJSTH5i/iBtqfMCT3L0pzEyFTNTNqjRDLJL99wYOm+QMTqONX+aZF5dO1ecp9wX7vT3ErSG+cvEy3oksByTC/l0c45Ez8gHB0aTVuRKgHi2F2cUve7Qq/bvbC1gE4bJOLBjxxiHhcEMt0Hxqceg9oaVv5vm6f23MEz1icxokeNgdAqIEF+ytvgs20uG1/v02zoIfycVLTrqFbksmwPqgg50ZA6VTS2ilwyJz2nAL05bdMEkF1GRmdDlrLzOjmD/2t/2/LwnVb8F2xlZw+qpoaZYwsvENtPG93TqOLchGW8qE4psBexKPw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679533413; bh=pJ3C+GVXrDpI2SaTMNBnmaguD4sQei9PupOMuoTcNFz=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=nmVpnxDiiF5WTl6BHD7U0u+AMHTfac/IjEjr3xtTezAxRvGNSi0jQEg9LqXeldUIcigiOBr+DT5OVJQyBxMiZdtmB4PBw8YgnM5k67WETUE9Ovn37MB4LVpqcCAFMXMBtcSwRKlG4p9NWFAX3nD2W8IP6eX7PXnzstOLFwNq0xbnHsBXbKHtyeMgGZNs3vWpdkWVTvyJQfMN0vkqL2KsawUs56n5B4gsmd4x9592XhNWS4rWsWgQsVHnmi7vmR+ptsvdCK7JO0oKDNaotcGYiFD/xvb/usz8DSA9hLpy81GCzWZEZi20aTUccdcUwOhfuOHUEV5wy1TrDnSFJnpSpQ== X-YMail-OSG: mcSh9lcVM1kLcOa7DqVF3M8e1e9_95oXMYXydcV4.OrtOMhVXu61Nch1ryJcfkx JPeBC1Cq44Dh2MDv_roEQsZ7vKotE82wzxQ_P_0l4H4ErZNz9sHBnj3bdKO5zXVI6KIDq_GSJDH4 vLbiKfF_ItXv4nfLTLksyaj7rAqossAIgbwQdGaRpUn6ssyXVHXVkTp5Eysm5VhNwsrkEnZBmg6O n.n4xrgr9U65Tut13TZXInNc5EdZ1ejF7Pll3oPE5I8t1auipaSd00GkHMg3sZnBpSJk_OW0j.K5 yS7FGapUq3nZ0LNUXOHo02bYut.7PfAOC11tGJzFeoJ7xwSwwHKVJcQ1b3P6oF9tZt90eDFBR0Jz yroS7nVa379KWQxfNN6Awukh_nzeIiLvzibeB8L6IDcVUtB0EGXLQ1CpVCGdHt6PXSBbDkcEz9kH H7K3w9JfonaqU5x08ZYlQyhiOHOKAKjElC3d0SJZjw5.u1r__XCp_rRm5RBYX3Kw9Ues4yz3aNMx vb1ywe8kRaAU4rnWAfrgYeDsxnq7I_t28zi8vYaWm_Z0jm4ChgZMYsCwhefP2XchKRSoyQ66dCKE hrvOu8vJe6G7O.KjHcKjzLvJ.CF_wG0BViiTEjnz2jGfB.9CpMLAiLZh8jIrxxXc35LdR596.jB3 fcJ.5_GLOHTfGNJJ_WQAJolu89E1VlX4NI.3xQGWKtaiqPrf_1j3LOhgIAhZgm.iJX29stdl.0vz KrurMeTT2K5wbsog7NhDtgQTgp6pgepTnzwaZe81nfCkDw1tKMUj9.YGJ2PeNhKq7vQuQ3E.CAXi dhsYs7xJjSbHX6DQ_dBfQB.LEFSUeHxgHwzcWnuCt2_w_cJbySkFtsrNnH1Jr9yON4UPPMPR7UeG oDqXXhWN1J0HF7_wGgs6eRtVBnQJ0gaBbRdVYN1tvH_SsvLcrtGf1fFI7t3B_B1LnVFSNCSOHyRP ikX3roMnWM9rBq9p4Ha18Ox_puGPFs9qxYvmpgjWLLeYYjgfZR8F2.tM1gmlnT33f44igW5u6bI5 o8eWyfFpWIPxoXZ53ZZ6xcOBhSRLJoBFHOubmAR9cg16jDLSgHdMgHqrESU4fYflc8_HoYLqEak8 ABU6PJ5X3wlKH9wEm7aCDz2hB1aRVJ6vVv2wdZbk2mt5HjHAQUy4pUHgoitQpYakqljtX2QCjtF_ Uf1fMlMB990j25svUiVdslyctx2CnbiR0kkkVmJb5oyXarIJjTPl9DSsElhONQUYCRTYWwMtk.p5 OkXIOHFNsG_fajdm26ymtUfnsX9sepijmJaHh2zkkO3sjn9MAwdbAgz1GHGd4JsExMaRbRzTGkr3 hKazKgLnE4ax5V03Q6tpqJuQVL9BTWsWA4suZECZ8Qri4nT3gUX0ckvuQIKiL874xKxtwXmx_5XH suiOAgt5xptHVeJeGNVCdCjbHMalUF7OYsPEaKnad_1A94ARCmk.2XQeswhfVZkxdhccckBD2tMD iMtcqHnidQgZU0LTowLywTaXjrz9c37_NP_9.pMpPmLBkpZOrAgMJimOy31S6vmF65VK1qdntjbe phe2u378FaD62YXoq5s3vwcavsxohDkfFxhrnThb2n20XzL.S6pk1Tj5y5r53HAPs0ECkR.GqLDy PIsdSD5VCF85Uh5WFi_L0J6ilFmSrniPvyZVw4PfFVjiZHk4yNQnOJgL1MgjYTs_gO6BlFdypygn 9pp4Ps4LzdKr3EXaeqUnqjTJmM163ZIDBG9mN9oIQfuLDtXxGMPkja4Y3gmhkVwX1wQe4l2S3g6v eOGGL2bZrz_8fG38SqvuDRgNNkRN5cNxUf2usTmx0BwTXNYp5iakenzpzNdgwkxBouVJ.2TeQtIu QWUt4LAk9RdHDUNBsLg_fWAb5SFa2OSmwzf9Fnh0j8ToNDMwElNKvNEBV9OP47n.jHMwHpSY8R6L wzXmYf9RAQm7eGuFRqxF1aR_Ob6mBBGpap0SyycO5iFrce7GO2FHzYbv0VW3ASCg4CWFpeQ4ve7c kG99f.A.8M0E.X0jDgpEXAKTygKXgBo.ksMCGNyZvcoXK9SWGT60IZ2E_USY37moVGt9mp35M6lk oF51lbLMI_SpszhrR84ms_R19QTsIUg23SOVu9Kk86vwK_kvtmQFwwZyCa8jrnu9om6BCtm3XmhV RHPSYSEKKoAzlaiSjgF4RcOiDyWu_GrZbQPPIzYAkLGccfnOVJWYYBjphTzJqnHNmu3vTPlmNN9D .Y3Ol_1d07wIDE9jVJ06Afyvjw5Qh63VCa.605XowIk0AlMALodpY8MwJarNNAEFY0vcuERCkGqD fRA-- X-Sonic-MF: X-Sonic-ID: f7bdf1f0-4f17-4259-b455-5e03658927b7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Thu, 23 Mar 2023 01:03:33 +0000 Received: by hermes--production-gq1-6cf7749bc8-kkr9m (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a4a676bf772f49a17ead82597b98981d; Thu, 23 Mar 2023 01:03:30 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: Periodic rant about SCHED_ULE From: Mark Millard In-Reply-To: Date: Wed, 22 Mar 2023 18:03:19 -0700 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: References: <6BD317F2-7EDD-45C0-9DC9-5B94C1BBB8E1.ref@yahoo.com> <6BD317F2-7EDD-45C0-9DC9-5B94C1BBB8E1@yahoo.com> <952d9795-19dc-8ad1-bb75-5c556ca6795a@m5p.com> <78EF511D-BAF1-495F-BAC9-03AC1B8FD56A@yahoo.com> To: George Mitchell X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TAGGED_RCPT(0.00)[freebsd]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.84:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.84:from] X-Rspamd-Queue-Id: 4PhnCX0tctz46J3 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Mar 22, 2023, at 16:17, Mark Millard wrote: > On Mar 22, 2023, at 15:39, Mark Millard wrote: >=20 >> On Mar 22, 2023, at 13:34, Mark Millard wrote: >>=20 >>> On Mar 22, 2023, at 12:40, George Mitchell = wrote: >>>=20 >>>> On 3/22/23 15:21, Mark Millard wrote: >>>>> George Mitchell wrote on >>>>> Date: Wed, 22 Mar 2023 17:36:39 UTC : >>>>> [...] >>>>>> Here are the very complicated instructions for reproducing the = problem: >>>>>> 1. Install and start misc/dnetc from ports. >>>>> Installing is likely easy, as likely would be building >>>>> with default options (if any). I know nothing about >>>>> starting misc/dnetc so that is research. (Possibly >>>>> trivial, although if it has alternatives to control >>>>> then I'd need to match that context too.) >>>>=20 >>>> service dnetc start >>>=20 >>> I built and installed misc/dnetc and got a binary >>> blob that clearly was not built in my environment: >>>=20 >>> # file /usr/local/distributed.net/dnetc >>> /usr/local/distributed.net/dnetc: ELF 64-bit LSB executable, x86-64, = version 1 (FreeBSD), statically linked, for FreeBSD 10.1 (1001515), = FreeBSD-style, stripped >>>=20 >>> Way older FreeBSD vintage than the locally available toolchains >>> would normally build. Some might be cautious about such a thing. >>>=20 >>> The man page reported that: >>>=20 >>> QUOTE >>> If you have never run the client before, it will initiate the = menu-driven >>> configuration. Save and quit when done, the configuration file = will be >>> saved in the same directory as the client. Now, simply restart the >>> client. =46rom that point on it will use the saved configuration. >>> END QUOTE >>>=20 >>> I've not seen what the configuration asks about yet. >>=20 >> I went through the configuration, basically just looking >> at it, other than providing an E-mail address. Then . . . >>=20 >> $ sudo service dnetc start >> Password: >> Cannot 'start' dnetc. Set dnetc_enable to YES in /etc/rc.conf or use = 'onestart' instead of 'start'. >>=20 >> $ sudo service dnetc onestart >>=20 >> I just let it run without any extra competing activity, other >> than I had my patched version of top running. It records and >> reports various maximum-observed (MaxObs) figures, here >> the load averages being relevant. >>=20 >> Top showed that dnetc started 32 processes, one per hardware >> thread. Mostly I saw: 100% nice and 0% idle. >>=20 >> Letting it run and then looking at the load averages (and >> their matching MaxObs figures) after something like 60+ min >> (not carefully timed: was doing other things) showed: >>=20 >> load averages: 31.97, 31.88, 31.66 MaxObs: 32.12, 31.97, 31.66 >>=20 >> (Note: The machine had been up for over 2.75 days before >> starting this and had not been building much of anything >> during that time.) >>=20 >> I've not yet experimented with having other, significant >> competing activity. >>=20 >>>>>> 2. Run "make buildworld". >>>>> So on the 32 hardware-thread (16 cores) amd64 machine that >>>>> I have access to, the test is to only have buildworld use >>>>> about one hardware thread, no matter what else is going on. >>>>> I never would have guessed that the steps would not involve >>>>> more like -j$(sysctl -n hw.ncpu) (so around -j32 in this >>>>> context). So it is good that you provided your note or >>>>> I'd not know if I'd done similarly or not when trying such. >>>>> [Note: -j1 and lack of -j are not strictly equivalent in >>>>> how make operates. As I remember, the distinction makes >>>>> a notable difference in the number of subprocesses created >>>>> directly by make (one per action "line" vs. one for the >>>>> whole block?). So even using -j1 might make a difference >>>>> vs. what you specified. I'd have to test to see.] >>>>=20 >>>> I am literally running "make buildworld" with no additional = options. >>>=20 >>> So required for repeating your results, but likely making >>> such results not be interesting relative to how I normally >>> deal with buildworld buildkernel and the likel, no matter >>> if there is other activity in an overlapping time frame or >>> not: my time preferences are too strong to wait for a single >>> hardware thread to do my normal builds, even with no >>> competing activity on the builder. >>>=20 >>>>>> Standard out conveniently reports how long it took (wall clock). >>>>> But nothing in your instructions indicate about how >>>>> to get an idea much progress dnetc made during the >>>>> various tests? [...] >>>>=20 >>>> Honestly, I've never worried about this part. But dnetc logs its >>>> progress in /usr/local/distributed.net/dnetc.txt, though not in = terms >>>> that are easy to relate to real-world progress. Oddly, when I run >>>> "make buildworld," I'm primarily interested in getting the world = built. >>>> Perhaps others feel differently. >>>=20 >>> Off topic for the specifics of the actual benchmark >>> that you run: >>>=20 >>> Then why not use of -jN ? In my context, any buildworld >>> using -j1 or no -j at all takes a huge amount of time >>> longer than letting it use all the hardware threads (or >>> so). (I've avoided having any I/O bound contexts for >>> such.) It does not take additional load on the system >>> for that to be true --including on the 4-core small arm >>> boards when I happen to buildworld on such (rare). >>>=20 >>>=20 >>>>> [...] >>>>> FYI: I've never built with and run the alternate >>>>> scheduler so if there is any appropriate background >>>>> for that that would not be obvious on finding basic >>>>> instructions, it would be appropriate to provide >>>>> such notes. >>>>> [...] >>>>=20 >>>> You have to build a new kernel, using a config file in which you = have >>>> replaced "options SCHED_ULE" with "options SCHED_4BSD". -- = George >>>=20 >>> Thanks for the notes. >>>=20 >>> I've not decided if I'll do anything with the binary >>> blob or not. >>=20 >=20 > FYI: >=20 > It is not your specific experiment, but I started my > "extra load" experimenst with . . . >=20 > I started a -j32 buildworld buildkernel with dnetc still > running. I'm generally seeing around 55% Active and 42% Note "Active": user, sorry. > nice, < 2% system (it was building libllvm at this point). > At that time: >=20 > load averages: 64.41, 60.52, 49.81 MaxObs: 64.47, 60.52, 49.81 >=20 Contrasting results for some obj-lib32 build activity: much more variety of User, nice, and system, including times with < 5% user, 90+% nice. But not typical overall. But lots of time roughly around 50%/50% or 35%/60%. There were times with 15+% system. Somewhat after buildkernel started: load averages: 69.15, 64.12, 58.72 MaxObs: 75.98, 64.12, 58.72 Harder to summarize, so overall timing reports from the buildworld and buildkernel stages. buildworld: -------------------------------------------------------------- ... World build completed on Wed Mar 22 16:37:57 PDT 2023 ... World built in 2615 seconds, ncpu: 32, make -j32 -------------------------------------------------------------- buildkernel: -------------------------------------------------------------- ... Kernel build for GENERIC-NODBG completed on Wed Mar 22 16:43:10 PDT = 2023 -------------------------------------------------------------- ... Kernel(s) GENERIC-NODBG built in 311 seconds, ncpu: 32, make -j32 -------------------------------------------------------------- Afterwards: load averages: 36.08, 53.14, 55.79 MaxObs: 75.98, 65.77, 59.84 I then did (not all in the same window): $ sudo service dnetc onestop # rm -fr /usr/obj/BUILDs/main-amd64-nodbg-clang-alt/usr/ before another -j32 buildworld buildkernel (no dnetc). The reuslts for this were: buildworld: -------------------------------------------------------------- ... World build completed on Wed Mar 22 17:39:19 PDT 2023 ... World built in 1240 seconds, ncpu: 32, make -j32 -------------------------------------------------------------- (compared to the 2615 for dnetc also in use) buildkernel: -------------------------------------------------------------- ... Kernel build for GENERIC-NODBG completed on Wed Mar 22 17:41:17 PDT = 2023 -------------------------------------------------------------- ... Kernel(s) GENERIC-NODBG built in 118 seconds, ncpu: 32, make -j32 -------------------------------------------------------------- (compared to the 311 for dnetc also in use) Experiments without -j32 will take a lot longer, even without dnetc in use. I'm not sure there will be such results today. =3D=3D=3D Mark Millard marklmi at yahoo.com