Date: Thu, 21 Dec 2000 13:58:59 -0800 (PST) From: John Baldwin <jhb@FreeBSD.org> To: Julian Elischer <julian@elischer.org> Cc: smp@FreeBSD.ORG, archie@FreeBSD.ORG Subject: Re: looking for locking advice.. Message-ID: <XFMail.001221135859.jhb@FreeBSD.org> In-Reply-To: <3A427757.8E9D0204@elischer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 21-Dec-00 Julian Elischer wrote: > John Baldwin wrote: >> >> Julian, >> >> Just use reader/writer locks for now. They can use a lockmgr backend until >> a >> more efficient reader/writer lock is implemented. As a side note, we should >> probably work up a rw lock API and define some macros that just map to >> lockmgr >> operations for now but can be optimized later. > > Having been looking at r/w locks for a week or two now I see several differnt > kinds of rw locks: > locks that sleep.. ok but us it efficient? > > locks that spin.. this is real tricky because interrupts > need to be blocked and since there are multiple readers you can't > assume that they finish in any particular order so you don't know > which interrupt state to restore. I gave up on this idea. > > locks that just return on failure to atain, and let the higher level > code decide what to do. We can always have this third (a nonblocking or try_enter()) item, while probably just using sleep locks that eventually can be implemented as adaptive (i.e. spin for a while, then sleep). > I have also for data going through netgraph r/w locks that on failure, > simply queue the packet for use later.. they never block and it > really looks sweet for that application. Yes, the best way to optimize is to optimize the bigger problem, not just the locking ops. :) >> As for mutexes: you should be >> using sleep mutexes, not spin mutexes, in which case interrupts aren't >> disabled >> while the lock is held. > > Fine, I just wasn't sure what the status of that was.. > what is the recommended way to 'sleep'? mtx_enter(&foo, MTX_DEF); will sleep if it blocks. >> Very, very, very, very few things should be using spin >> mutexes. > > I spin for my r/w node locks but I think that on averae they'll be spinning > for > about 10 cycles as I'm only serialising VERY short operations. The sched_lock mutex is used to protect parts of the mutex code for sleep mutexes, so implementing rw locks is a place where a spin lock might make sense. The other place it makes sense is in a "true" interrupt context for fast interrupts. In fact, it is mandated in those cases, because we can't block in a "true" interrupt context. >> Note that when a mutex is released, if any threads are blocked on it, >> then the thread with the highest priority is woken up so it can run. Also >> note >> that interrupt threads have a higher priority than all other threads, so >> yes, >> you can block in an interrupt thread. > > good.. is there priority inherritence for the fellow that holds the lock I'm > waiting for? Well, yes, it is there, but no, it doesn't work right at the moment (just hangs an SMP box) so it is turned off atm. >> However, blocking an interrupt thread >> is something that should still be somewhat minimized due to shared >> interrupts >> since you will be blocking all interrupt handlers associated with a >> particular >> interrupt source, thus don't go using tsleep() in an interrupt handler, for >> example. :) > > That's understood.. I presume I can't get another of the same interrupt > while processing the last one? Actually, you can, but all an interrupt causes to happen is that the associated ithread gets scheduled to run if need be, and is marked as pending (it_need in struct ithd). The ithread will call its handlers in a loop as long as it_need is non-zero (zero'ing it on each pass through the loop). > All I really want to do is ensure that if I am updating a tree or hash-table, > that no-one else is navigating that structure at that time. The updates > should > be very fast (insert/delete single node), and the navigations should be > equally > fast. If they are very fast, then a simple mutex lock will probably be fine. A rw lock is most useful when the operations take a little while (you call a complex function on each item in the list for example, as opposed to just walking the list doing comparisons) and you want to allow concurrent readers. > The optimum action would be: > spin if it's another processor you are waiting on (and it's running), and > yield > to it if it's the same processor, and you've pre-empted it. Unfortunatly > I don't think that is practical at this time. Maybe a simple version would be > the > "spin for 100 cycles and then sleep" version.. (if it were an MP machine) > > Maybe locks should have 'processor held by' as one of their fields.. > I guess you could get it indirectly via the proc structure..... Sleep locks are owned by a process, not really a processor since a thread can block on a mutex on cpu A, wake up and obtain it on cpu B, block on another mutex, wake up and obtain that mutex on cpu C, get switched to A after using up its timeslice, and then release both locks on A. Spin locks are owned by a process, too, but you can't do a context switch while holding a spin lock (except for sched_lock in cpu_switch()), so they effectively are owned by the CPU holding them, which can be found via curproc->p_oncpu in the SMP case. The "plan", as it were, is for MTX_DEF mutexes to eventually become adapative, which will hopefully get us closer to optimum performance without having to do too many extra checks or other overhead. -- John Baldwin <jhb@FreeBSD.org> -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.001221135859.jhb>