Date: Tue, 29 Jun 1999 01:16:51 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: dillon@apollo.backplane.com (Matthew Dillon) Cc: tlambert@primenet.com, freebsd-smp@FreeBSD.ORG Subject: Re: high-efficiency SMP locks - submission for review Message-ID: <199906290116.SAA08367@usr05.primenet.com> In-Reply-To: <199906280742.AAA18132@apollo.backplane.com> from "Matthew Dillon" at Jun 28, 99 00:42:37 am
next in thread | previous in thread | raw e-mail | index | archive | help
> : In particular, the use of a single wait count means that you > : will not have an ordered list for the equivalent of a "wake > : one". This is a real problem, since it allows for a deadly > : embrace deadlock to occur when two kernel contexts each hold > : one lock and want to acquire the other contexts lock. > : > : I believe that resolving this as an EWOULDBLOCK, and then > > Well, I differ with you here. lockmgr() can't even handle that. Well, if you are going to throw out the baby, you might as well throw out the bathwater while you are at it. 8-). > Lock primitives that become too complex can no longer be categorized > as primitives. That is one of the biggest problems with lockmgr(), > in fact. OK. By this definitoin, I would categorize my suggestions as higher level ocking facilities, which should be built on primitives. The big issue to address in the primitives themselves is support for intention modes. > What should be done instead is to separate the functionality of the > more complex locking functions - such as those that deal with deadlock > situations - because these locking functions have a considerable amount > of overhead compared to lower level primitives. I disagree, but that's more of a where-and-what-you-lock argument than we need to get into right this second. > : Finally, there is insufficient mechanism to avoid competition > : starvation, where a write blocks indefinitely as multiple > : readers pass the resource between each other. > > Yes, this is an issue -- but the solution in lockmgr() has only led > to more esoteric deadlock situations and, I think, harmed performance > as much as it has helped it. lockmgr() is a hulking behemoth that does what it was intended to do well; unfortunately, it's being abused for all sorts of strange and not-so-wonderful uses, and it's not graceful about boundary conditions or error recovery. > That isn't to say that we can't implement > the same solution in qlocks, but the way I would do it would be by adding > a new function, not modifying existing primitives. For example, > qlock_wr() might not try to hold-off other shared locks, but > qlock_hipri_wr() could. OK. My main issue is that if there's going to be rearchitecting of the locking facilities, that it be done in a comprehensive fashion. This doesn't mean that every has to be implemented, but that it needs to be thought out in order to not _preclude_ everything from being implemented in the future. I think that there is a need for someone to get their head around the whole thing, and write it down, before it degrades into "change for the sake of change" when someone goes in and "optimizes" code that was written to intentionally allow work in future directions. I'm not accusing you of this (obviously; the code isn't even in yet!), but it's something that should be considered in detail before acting. > The biggest problem we face at the moment is that locks are being held > for much too long a period of time. For example, locks are being held > *through* I/O operations as a means of controlling access. This is > precisely the wrong way to use a lock. The lock should be used to > protect the data structure and then released. An I/O operation in > progress should set a "the data is being messed with" flag and then > release its lock, not attempt to hold the lock permanently. Or, as you > mentioned, intention locks can be used to separate the I/O op from other > types of operations. I think that this type of thing could be fixed without introducing the variable of new primatives to contend with at the same time. This is a song I've sung before, with regard to not introducing intential VM aliases until the unintnetional ones have been eliminated, etc., so I won't sing it longer than this paragraph. > Many of the shared/exclusive problems go away ( or at least go into > hiding ) when the locks are used properly. A bug hiding is worse than it shoving its cold nose into your butt unexpectedly while you are busy doing something else... I think that anything that might cause something to "go into hiding" is inherently evil. 8-(. > : I believe the following is the minimal set of structures > : required to resolve the blocking operation inheritance and > : deadlock detection: [ ... ] > Holy cow, that is very expensive. If I were to implement that sort > of locking subsystem it would be at a higher level, it would not > be a primitive. Agreed. However, I have experience with something similar from back in 1993/1994, and I can tell you this: 20,000 transactions a second on a 66MHz Pentium. > I tend to dislike complex locking solutions, but only because I > strongly believe that it is possible to avoid the necessity of > such by organizing code and algorithms properly. Me too; however, you are unlikely to be able to shove in the necessary rearchitecture to avoid holding context across function call boundaries. If you manage to somehow do this, I expect a crucifixtion or two... > As lockmgr() has > shown, trying to implement complex locking solutions in an SMP > environment can become *very* expensive. So expensive that the > complex locking solution winds up hurting performance more then > a simpler solution would help performance. The lockmgr() is, and always has been, too heavy weight for decent SMP performance. > You are right on the money here. This is precisely the problem > that the SMP implementation currently faces and is attempting > to solve with spl-aware simple locks. In fact, the way spl*() ops > work currently is very similar to obtaining multiple locks > simultaniously. Again, I really hate having to implement complex > solutions when it may be possible to obtain the same effect by > reorganizing the functions that require the locking in the first place. Complexity is a function of observation. For something like lazy task context creation for blocking, it's not really that complex. It's about the same order as a call conversion scheduler, or the use of multiple client contexts and a select loop. It's a matter of discipline (and documentation). I really have seen very few real situations in which threads were truly advantageous compared to implementing the code with implicit or explicit internal scheduling, for example. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199906290116.SAA08367>