From nobody Sat Mar 25 18:14:11 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PkS084ThXz41XRr for ; Sat, 25 Mar 2023 18:14:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-20.consmr.mail.gq1.yahoo.com (sonic305-20.consmr.mail.gq1.yahoo.com [98.137.64.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PkS072GpMz3Drx for ; Sat, 25 Mar 2023 18:14:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=akgJgXDa; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679768068; bh=sBysVb4XFx4ipjzfGmlmR2au+qgUbz+Uo4p6lRKjUhs=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=akgJgXDa5ajtR+rsPaAbuZI+4qig8K23wU8gjkv74Ea8RTxMKR+fEYvu6ThGSz8fPERCZc3Y/LDvjH34onmRPyTJ0bQZvO/FFxB+V50vOl9MR13pi/x02Mdks5F/dJaCHQmcJDcHhT6hBmluzcsjwCuEYfctZUooAduxMs3l5fKhqtEwJTa4+tufSn7WwPhAP9SdyCE1HoFXoTY2v+LVpWU18UIovqMGb3Nyo7omb6trwAjO7y1ZzLVWOI3gjdxpg+BJ0WyhBm+D+0uNPZnA+uC4w03fNvNcNPvaGGzDCc/h1uku4mkdn2fBlbTUoUJVU5OxjhkGujsgptagiDGnuw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1679768068; bh=iCzwFUbEqxOHlZAlW/YL1GVWgUoVEh0j8X9f06iaS6P=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=kwwZBwh0omu0IkVMqsvLKqSlQ2RTY0V8oAchmaCfNaMWbEhzXfYQ0mNqTO167QGHSCxtIBFKbxZzSnnml6WzVvHrN3Th8byEUfE38tSiV2zCOshvUWy/Lkk7NRrtPntpAlcQ2ljBDBJ1J2Whydd18W1/ywYHT5KsXYufi0CjJ4927YRUeZPAq8OdU54+u3YpU7SgnilaaSaxJATF+D30yXwXLfzrX5zAWjKlBUm8LgqkxjaLUvESUo1Yv6L880ysjsZ4mDxJJ4wn5THs6A2sTiCT+JMJLpeOA6/O5An9yHnXWN9djwJeV2JFVJVU6bySNHEoV8QmEi1L6GcenpgJIQ== X-YMail-OSG: YfNc3MEVM1mswA5ngZzxzngtQAJc.ZReSLTXN.PaDBJpwrKO0Gi6XeIpaGKgXLa SkSv0LbA4qeFdAqsM8gXH_BY8TjW.Mb8X78DcMm1f_kegQ7k_Wgrl0Pwj8O5VI5WdTAgMNt6uqq7 ERZm6jYIsXJiMSSg0wVk4Bs4CG2NlwhwKKwVtat5eG5Z9AwyA.WdJ2sYKn_G3WWVMKovgwnJrGAr jq0heKEfId37gjkeOsugFOyR5bPw96_.Kc80lZs4DPaSAL2h9xgVmjxtsqRLV4eqZPmtWTL64bs4 tiFOJDGUGO2g72GbaTo4rnGSZtyX3Bvxmn2bzSSVDp_dNOERZR2jNkLUKO25Rbgc7s3a6S8O4mUj 7LspP6B23ncFOwQrVLkYi2oBSK1jtnkTuIAcu12Ot1XFjzflaIx2Ht1EDS32H_jaSeXB60cCkL6a zLDrgbTJxIbNy3ruGi4kM3s6vX2y9Ouz8ujn6WkiWREJnTVJdztAdwXeyrXpv9K1TSpCN_jo9Ytq AtnxxvK4wtmnGUAbHzzwejC8ix.VgL2uAILr.vP1DZEHjAZzWnK_MHl4BUp7nQju8wC_4CG_Km3r JeSfFUMS3hYRJEQ3xkqjBzOfxuKvFY9HUIwBM9_NQlm1HHcvABfUPHT9cqPaYp9Lrq.apYDUME_F pe3Di.vL6fJxp22eP1SujdeL7tkfPZirqXec.ftE8mOV9dy4zjoJRflzHHujvev7_BhMlKlI0GFg f_DQz.lZmyuk1gPrBZixY1ISDm7JzBscMGvlquF6.qJf6cwp4Ly8sfoH.Q9JNDG7cLMbOezsc3hB 3mCO2QuPyiKM5FaqZF6I_qEC0iXcjoGVqfLQCB.Cyn951rYt0MH8Ix4Em55jDFaGk8LysONU45o6 Wt5cwHEwK2LOBgGwVxLrGKqLabQHMGV.BcPsBaR0e4NjaTjXvEppQ7pqhp2YyCSzswMz_r8yDiD3 MnGcmz3oCE3djrM9B.JJq8Y8.RURK9.yIA40Vn4pvWz7N7EX55IZZyVxyyl_Yl4nWryYcHJTlKL9 8ZFEaHnJAgI9TUG5N1xLRv2jIprbi0sPVJiUidjY_hedN4NM9GXZTNVRTE0IxZwkg6F65J4x6Hhn zHsb1XvR_NJJwN5Kin0ACgCJUFq5S4J04miJr.71tzsu_vqlwqL66EZl.yWaTnDRp6S5KG_Btf2A 7pX4y_5jpPgypgNDc5gJcVToWrPPoIn6jV6CUCdiDe3hwIVFi1Gy4VIOHKGl4t9CpkAPaaGu_MZd Az2b5Qmag7VNAm5Tz3Woz2YgYU0rA99eY0knW9dS4ey77BwUWvfKiCxO1KE.P6JK95ZXa.m2qamc FumRICG0BOO9I6fA_68CnZaMLSDFkCxLrNzPt0xIp9F2ScWGJsvhqsUk0j4zG744Bsdze8ZyV7SN u11P4CmeEUDxdVYeHqiTETbjytDIIRPYfAOkW6fgDpr_QC4yktNZk1MkrnHykOHAdCaOPTUtjG3t QHFGmmN3ASIH5VHa5DSwBcUpcjtVFUB5uuVjE_mjsFdH5UZrExFX5aZCYooo6HGEbom8rUCJGMJe tDwiR..JfTgXHnXnqjgbwehUBK4Sib9ENbYr6itO2NskbkSH7rR0wt1P4tModazbYI0qadLoAVWo Fb0UW.cOnzhvrOLVkMwZgHQj4gkaOzd2FJvMzGpEEmY2wHtFwuXSNJhbsKqwCyBAFNf9K18A7jiq jE1UOP8WkybOp8HfuBMhKzyWk1Zo2i.CvhQ29x7aHE050CUyTcg0VUsi_J2Gy47mycULeZK3jPYk z_pEItB4NvRWxZHMPLCmMqmX3swtyYp2RN1us60chhLjjQkkgFB5j_ENoZyYPh1gl5.bWDM6sy5h 9vlqajcx3q0E8RBkSALzGaTPC2hLADR1JZQCKnCERTc8oa6gr8O_BRpy2tCAwCN7mIMtv0wDM8GJ 6JyDcBGkHTB2ys6lZS.Jhd8ENMzAfKlTOpYvSFURYcWkBRdO5OvwCt8UqKIDRWDG3C275y0FqEUF EXZhDa.XrljvuH.i30vtK7LXhJYxgoNzg6T8Bmah74ksFfwYBhUy9LcYqbD3n_Zfl4UdXqhsZaXd pXjPcjf_Vaylbt.YsPIz0pubCFS2Y5h1QmSG6RxOUsOTeq8JD.XVWjEWOsMNVdUFQLJLMhDPbQ5u .EN8Aoq9BIBIRYYQ8LnReXeoS2jOnIpqr79gwxHXrxlxIcipqyi_Ifl5kYf4Gv4BTFiHC1IBPm0K cyMS_iOB93P4KaDDwkC1UUkB6s_942bAavsNFglOVQFre.EIb42yeyiZds1Qj4ot61uPEDe.Wjq7 9pw-- X-Sonic-MF: X-Sonic-ID: 53c96eb3-27d6-45c5-a63d-74ffd1ef389a Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Sat, 25 Mar 2023 18:14:28 +0000 Received: by hermes--production-bf1-777648578f-75chz (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 3af61c8aca561d0e34e3ab64f6f8428d; Sat, 25 Mar 2023 18:14:23 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: Periodic rant about SCHED_ULE Message-Id: <5AF26266-5B4C-4A7F-8784-4C6308B6C5CA@yahoo.com> Date: Sat, 25 Mar 2023 11:14:11 -0700 To: Peter , FreeBSD Hackers X-Mailer: Apple Mail (2.3731.400.51.1.1) References: <5AF26266-5B4C-4A7F-8784-4C6308B6C5CA.ref@yahoo.com> X-Spamd-Result: default: False [-3.30 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.80)[-0.796]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.83:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org] X-Rspamd-Queue-Id: 4PkS072GpMz3Drx X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Peter wrote on Date: Sat, 25 Mar 2023 15:47:42 UTC : > Quoting George Mitchell : >=20 > >> = https://forums.freebsd.org/threads/what-is-sysctl-kern-sched-preempt_thres= h.85 > >> > >Thank you! -- George >=20 > You're welcome. Can I get a success/failure report? >=20 >=20 > --------------------------------------------------------------------- > >> On 3/22/23, Steve Kargl wrote: > >>> > >>> I reported the issue with ULE some 15 to 20 years ago. >=20 > Can I get the PR number, please? >=20 >=20 > --------------------------------------------------------------------- > Test usecase: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > Create two compute tasks competing for the same -otherwise unused- = core,=20 > one without, one with syscalls:=20 >=20 > # cpuset -l 13 sh -c "while true; do :; done" &=20 > # tar cvf - / | cpuset -l 13 gzip -9 > /dev/null=20 >=20 > Within a few seconds the two task are balanced, running at nearly the=20= > same PRI and using each 50% of the core:=20 >=20 > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND=20 > 5166 root 1 88 0 13M 3264K RUN 13 9:23 51.65% sh=20 > 10675 root 1 87 0 13M 3740K CPU13 13 1:30 48.57% gzip=20 >=20 > This changes when the tar reaches /usr/include with it's many small=20 > files. Now smaller blocks are delivered to gzip, it does more=20 > syscalls, and things get ugly:=20 >=20 > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND=20 > 5166 root 1 94 0 13M 3264K RUN 13 18:07 95.10% sh=20 > 19028 root 1 81 0 13M 3740K CPU13 13 1:23 4.87% gzip=20 Why did PID 10675 change to 19028? > This does not happen because tar would be slow in moving data to=20 > gzip: tar reads from SSD, or more likely from ARC, and this is=20 > always faster than gzip-9. The imbalance is made by the scheduler. When I tried that tar line, I get lots of output to stderr: # tar cvf - / | cpuset -l 13 gzip -9 > /dev/null tar: Removing leading '/' from member names a . a root a wrkdirs a bin a usr . . . Was that an intentional part of the test? To avoid this I used: # tar cvf - / 2>/dev/null | cpuset -l 13 gzip -9 2>&1 > /dev/null At which point I get the likes of: 17129 root 1 68 0 14192Ki 3628Ki RUN 13 0:20 = 3.95% gzip -9 17128 root 1 20 0 58300Ki 13880Ki pipdwt 18 0:00 = 0.27% tar cvf - / (bsdtar) 17097 root 1 133 0 13364Ki 3060Ki CPU13 13 8:05 = 95.93% sh -c while true; do :; done up front. For reference, I also see the likes of the following from "gstat -spod" (it is a root on ZFS context with PCIe Optane media): dT: 1.063s w: 1.000s L(q) ops/s r/s kB kBps ms/r w/s kB kBps ms/w = d/s kB kBps ms/d o/s ms/o %busy Name . . . 0 68 68 14 937 0.0 0 0 0 0.0 = 0 0 0 0.0 0 0.0 0.1| nvd2 . . . =3D=3D=3D Mark Millard marklmi at yahoo.com