Date: Fri, 11 May 2007 18:23:03 +0200 From: "Attilio Rao" <attilio@freebsd.org> To: "Julian Elischer" <julian@elischer.org> Cc: smp@freebsd.org, marko.kiiskila@nokia.com, Giri.Rashiyamany@nokia.com, jeff@freebsd.org Subject: Re: threadlock patch Message-ID: <3bbf2fe10705110923g2f072d58ld2d15f5eddd0684f@mail.gmail.com> In-Reply-To: <46438B80.6000404@elischer.org> References: <2BC1E09BAED3A94ABF23A2B2469D1E59739C02@esebe108.NOE.Nokia.com> <4640CDFA.9090602@elischer.org> <0AC25B27A4FD2E4B839C9FAF2C704DBA012E79D4@daebe103.NOE.Nokia.com> <20070510123608.N643@10.0.0.1> <46438B80.6000404@elischer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
2007/5/10, Julian Elischer <julian@elischer.org>: > Marko, > We are just reorganising our effort on this stuff > to put it in a bit more of an organised manner. > > for a start.. there is a perforce branch that covers this work.. > > http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=/depot/user/attilio/attilio%5fschedlock&HIDEDEL=NO > > (hopefully that didn't wrap) > also we are using the mailing list smp@freebsd.org > to discuss this. I suggest that you and any of the guys there who are intersted in > pushing this sign up to catch the fun. I've CC'd this to the list.. > I suggest we use that list to communicate so that there is a record. > also makes is easier for others to follow and join when needed. So here we go. With this e-mail I'd like to explain the state of the sched_lock pushdown work and its implications. Start with saying that even if sched_lock has showed to be the most conteded lock in our kernel, effort for breaking it is intended to serve mainly for ULE scheduler optimizations. Having nomore a global lock, infact, will leave us splitting the current global runqueue into per-cpu/per-core runqueues optimizing a lot ULE behaviour. The approach choosen for breaking sched_lock is, IIRC, what Solaris alredy uses in its kernel: using containers lock. Basically, every "container" (as sleepqueues, runqueues, turnstile queues, etc.) has its own global lock. Every thread holds a ptr to the specific container in which is working and so thread locking results in locking the container. This allow an extremely high parallelism and a avoids a lot of pain with LORs (that could be raised by different approaches). In a perfect world this would have been enough to accomplish the task, but in this world is not; basically, sched_lock is currently used in the kernel in order to protect other things like per-proc flags, like ldt locking (in ia32), etc. In order to lock these other parts alternative approaches have been investigated. So, the work has been basically divided into two different parts: 1) sched_lock substitution in 'miscellaneous' subsystems 2) implementation of the container locks handler That perforce branch tries to address rather completely the problem n.1 while Jeff worked mainly on the problem n.2. More specifically, the branch contains: - locking for ldt in FreeBSD-ia32 - locking for global vmmeter cnt - locking for timer - locking for rusage The idea is to have the branch to be well working and well tested (and well benchmarked, as well) and possibily to commit into CVS these changes before to go on and integrate Jeff's work, since it is the most important and we want to be sure to work on a strong base. Currently, open problems are: - approach for locking rusage should be better evaluated and different approaches should be tried on either UP and SMP systems in order to get the better one - a first benchmark on Jeff's patch (using sysbench), showed an higher peak (IIRC, between 5% and 10%) on 8-CPUs machine, but the scalability got a lot worse. We have to understand why the scalability suddenly breaks down and resolve that. I hope all you have a clearer situation now, if you want to know more details just please write an e-mail :) Thaks, Attilio -- Peace can only be achieved by understanding - A. Einstein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3bbf2fe10705110923g2f072d58ld2d15f5eddd0684f>