From nobody Wed Mar 22 19:04:06 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PhdDn1tz9z419q8 for ; Wed, 22 Mar 2023 19:04:09 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.95.76.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "troutmask", Issuer "troutmask" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PhdDm6X2Vz45PF; Wed, 22 Mar 2023 19:04:08 +0000 (UTC) (envelope-from sgk@troutmask.apl.washington.edu) Authentication-Results: mx1.freebsd.org; none Received: from troutmask.apl.washington.edu (localhost [127.0.0.1]) by troutmask.apl.washington.edu (8.17.1/8.17.1) with ESMTPS id 32MJ47uq001819 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 22 Mar 2023 12:04:07 -0700 (PDT) (envelope-from sgk@troutmask.apl.washington.edu) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.17.1/8.17.1/Submit) id 32MJ47KZ001818; Wed, 22 Mar 2023 12:04:07 -0700 (PDT) (envelope-from sgk) Date: Wed, 22 Mar 2023 12:04:06 -0700 From: Steve Kargl To: Matthias Andree Cc: freebsd-hackers@freebsd.org Subject: Re: Periodic rant about SCHED_ULE Message-ID: Reply-To: sgk@troutmask.apl.washington.edu References: <8173cc7e-e934-dd5c-312a-1dfa886941aa@FreeBSD.org> <8cfdb951-9b1f-ecd3-2291-7a528e1b042c@m5p.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4PhdDm6X2Vz45PF X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:73, ipnet:128.95.0.0/16, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Wed, Mar 22, 2023 at 07:31:57PM +0100, Matthias Andree wrote: > > Yes, there are reports that FreeBSD is not responsive by default - but this > may make it get overall better throughput at the expense of responsiveness, > because it might be doing fewer context switches. So just complaining about > a longer buildworld without seeing how much dnetc did in the same wallclock > time period is useless. Periodic rant's don't fix this lack of information. > I reported the issue with ULE some 15 to 20 years ago. I gave up reporting the issue. The individuals with the requisite skills to hack on ULE did not; and yes, I lack those skills. The path of least resistance is to use 4BSD. % cat a.f90 ! ! Silly numerically intensive computation. ! program foo implicit none integer, parameter :: m = 200, n = 1000, dp = kind(1.d0) integer i real(dp) x real(dp), allocatable :: a(:,:), b(:,:), c(:,:) call random_init(.true., .true.) allocate(a(n,n), b(n,n)) do i = 1, m call random_number(a) call random_number(b) c = matmul(a,b) x = sum(c) if (x < 0) stop 'Whoops' end do end program foo % gfortran11 -o z -O3 -march=native a.f90 % time ./z 42.16 real 42.04 user 0.09 sys % cat foo #! /bin/csh # # Launch NCPU+1 images with a 1 second delay # foreach i (1 2 3 4 5 6 7 8 9) ./z & sleep 1 end % ./foo In another xterm, you can watch the 9 images. % top st pid: 1709; load averages: 4.90, 1.61, 0.79 up 0+00:56:46 11:43:01 74 processes: 10 running, 64 sleeping CPU: 99.9% user, 0.0% nice, 0.1% system, 0.0% interrupt, 0.0% idle Mem: 369M Active, 187M Inact, 240K Laundry, 889M Wired, 546M Buf, 14G Free Swap: 16G Total, 16G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME CPU COMMAND 1699 kargl 1 56 0 68M 35M RUN 3 0:41 92.60% z 1701 kargl 1 56 0 68M 35M RUN 0 0:41 92.33% z 1689 kargl 1 56 0 68M 35M CPU5 5 0:47 91.63% z 1691 kargl 1 56 0 68M 35M CPU0 0 0:45 89.91% z 1695 kargl 1 56 0 68M 35M CPU2 2 0:43 88.56% z 1697 kargl 1 56 0 68M 35M CPU6 6 0:42 88.48% z 1705 kargl 1 55 0 68M 35M CPU1 1 0:39 88.12% z 1703 kargl 1 56 0 68M 35M CPU4 4 0:39 87.86% z 1693 kargl 1 56 0 68M 35M CPU7 7 0:45 78.12% z With 4BSD, you see the ./z's with 80% or greater CPU. All the ./z's exit after 55-ish seconds. If you try this experiment on ULE, you'll get NCPU-1 ./z's with nearly 99% CPU and 2 ./z's with something like 45-ish% as the two images ping-pong on one cpu. Back when I was testing ULE vs 4BSD, this was/is due to ULE's cpu affinity where processes never migrate to another cpu. Admittedly, this was several years ago. Maybe ULE has gotten better, but George's rant seems to suggest otherwise. -- Steve