Date: Thu, 24 Jan 2019 13:10:59 -0500 From: Zaphod Beeblebrox <zbeeble@gmail.com> To: Jan Bramkamp <crest@rlwinm.de> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Scheduled ideas: Two threads with one "stone" Message-ID: <CACpH0MctO3mDm0oNfvY0OAzydw1hu4OUk5AoBpY_DGze4q4QXw@mail.gmail.com> In-Reply-To: <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de> References: <CACpH0Mc09tjybFKpdh=yMRKxvMuHjRC4k6sen1kUo2AoLd7UHw@mail.gmail.com> <1ce4898f-942b-d5ec-7563-fdceb56f4e88@rlwinm.de>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jan 24, 2019 at 12:36 PM Jan Bramkamp <crest@rlwinm.de> wrote: > On 24.01.19 17:32, Zaphod Beeblebrox wrote: > > W.R.T. hyperthreading, it's come to my attention that FreeBSD has had > some > > recommendations for awhile --- namely hyperthreading is bad for secure > > systems. That-is-to-say this is not about recent vulnerabilities of > recent > > Intel flavoured CPUs, but rather some thoughts on a different solution > to a > > different problem. > > > > I was building world on my laptop when I noticed how sucky it had become. > > Now... I well know this is due to the fact that even a "nice -19 make > -j32 > > buildworld" will slow the normal priority userland down by about 1/2 > > because the thread that is "me" on the CPU is competing with one of the > > threads that is saturating the CPU. My normal priority thread will get > > scheduled, but it competes equally for resources with threads at much > lower > > priority. > > You can use `idprio 10 ...` to minimize the impact of CPU bound tasks. > If you want to go even further you could apply hierarchical resource > limits on the make process. The idprio chainloader puts make and all its > descendants into the idle scheduling class which allows normal time > sharing threads to preempt them. > No. This is insufficient to the task. If your goal is that the "idle" or "low priority" task does not effect the high priority tasks, this is insufficient w.r.t. hyperthreading. In hyperthreading (to discuss just one variant of many), two CPU cores share all of their functional bits. This is useful because, in general, CPU functional bits are fairly idle. Even if both threads on hyperthreaded pair are "runable" right now, they can be fetching from memory or, in the exreme, one could be issuing floating point and one could be issuing integer, IIRC. The problem I'm proposing to solve is when the two processes are of sufficiently differing effective priorities (however they get to that point) that they not be scheduled on the same hyperthreaded pair. The reason for this is that beyond scheduling the processes to the cores, we have no control over the resource contention of the cores --- and by that I claim that the lower priority process can force the higher priority process to wait as it consumes resources shared by the two cores. In an ideal world, the hyperthread interface on the cpu would allow some combination of prioritizing cores and their resource use, but that is not how Intel's chips currently work. To be clear, and this is easy to experience, find a FreeBSD desktop with a i7 (or similar) running X and (say) kde. Not something lightweight. Make sure it has enough RAM (16 or 32 big is probably good for this experiment). When idle, take a "feel" on the interface. No lag, things are spiffy. Now, go to /usr/src. Make sure it's there. Then run "nice -19 make -j32 buildworld" there. For extra credit, you can even use idprio if you like. Now... duplicate your feel test. The interface will be laggy. Things will be slower. If you look at the process table, it's not that your effective priorities are coming anywhere close to those from -19 ... or even if you use idprio... still laggy. What is happening here? Think about your processor. FreeBSD will ensure that your normal priority processes are scheduled first on a few of the 8 cores that show with an i7. But chances are very high that your processes are landing opposite low priority processes on the hyperthreaded pairs. This means that your effective processsor use is being cut in half (-ish) by the hyperthreaded processes grabbing the shared resources roughly half the time. You can game out having two processes ready and how often they fall on the same pair vs. falling on non-paired. You can even do the thought experiment of higher priority processes being placed on cores with pairs deliberately not used until all non-pared cores are in use --- urm... this is a big diversion... then we have the opportunity cost of the then partially unused resources ... etc. I should stop there. There are simple things that make a big difference before that digression produces many small things with even smaller effects.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACpH0MctO3mDm0oNfvY0OAzydw1hu4OUk5AoBpY_DGze4q4QXw>