Date: Sat, 5 Jun 2004 22:55:31 +0200 From: "Ali Niknam" <ali@transip.nl> To: "Robert Watson" <rwatson@freebsd.org> Cc: freebsd-hackers@freebsd.org Subject: Re: FreeBSD 5.2.1: Mutex/Spinlock starvation? Message-ID: <00da01c44b3f$74d3d8c0$0400a8c0@redguy> References: <Pine.NEB.3.96L.1040604153442.34555O-100000@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Robert, As promised my findings regarding the changes; just came home after a night of trying and praying :) > Actually, by default, most mutexes in the system are sleep mutexes, so > they sleep on contention rather than spinning. In some cases, this > actually hurts more than spinning, because if the mutex is released > quickly by the holder, then you pay the context switches which cost > more than spinning for the short period of time. > > You might want to try adding "options ADAPTIVE_MUTEXES" to your kernel > configuration, which will cause mutexes to spin briefly on SMP systems > before sleeping, and has been observed to improve performance quite a > bit. > I tried this; this helps performance a lot, here are the findings: - under all conditions turning on HTT helps a *lot* (which is logical i think) - under non killing load (killing load = load where server would have crashed without this option) it performs much much better - under killing load it performs a lot better, up until a certain level: - a new killing level: from this point onward basically the same thing happens as before.. What i'm guessing is that probably this new killing level occurs when load is so high that the spins 'adapt' into blocks. From your description above I understand that there's a certain timeout when 'spinning' mutexes turn into 'blocking'/'sleeping' mutexes. Is there a way to set this timeout ? I would very much like to try out what would happen if one would set this timeout to a quite high value. Appart from this i also tried options ZERO_COPY_SOCKET, but that didnt seem to help much, if at all. Furthermore I tried out SCHED_ULE which was dramatic! I'm not sure if i'm the only one, but the performance was really terrible. i switched it off again as soon as i could. Also what I was wondering: do processes that go into sleep-mutex mode go into the same waiting queue as normal processes, or do they go into a special queue? Could this problem basically boil down to a scheduler being to slow (or the context switching) for these amounts of processes waiting/blocking ? If so could it be an idea to put blocking processes into a special queue in which the scheduler adepts simple scheduling algorithm (such as first come first serve, or random, or whatever) to dramatically reduce rescheduling time ? Best Regards, Ali Niknam
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?00da01c44b3f$74d3d8c0$0400a8c0>