Date: Wed, 5 Jun 1996 11:00:39 -0700 (MST) From: Terry Lambert <terry@lambert.org> To: michaelv@HeadCandy.com (Michael L. VanLoon -- HeadCandy.com) Cc: terry@lambert.org, smp@freebsd.org Subject: Re: Unix/NT synchronization model (was: SMP progress?) Message-ID: <199606051800.LAA29231@phaeton.artisoft.com> In-Reply-To: <199606050519.WAA25284@MindBender.HeadCandy.com> from "Michael L. VanLoon -- HeadCandy.com" at Jun 4, 96 10:19:45 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> >> Early SMP versions of just about every SMP Unix OS I know of started out > >> being a single lock and became fine-grained in subsequant versions (in many > >> cases before the code was officially released). > > >Yes. The simplest method of increasing granularity is to tier the > >locks as global mutexes (not fix that yet), and then "push down" the > >locks through th trap code into the system call layer. > > What exactly do you mean by "push down the locks through the trap > code"? > > >At that point, you are free to add locks to structures accessed as a > >result of an interrupt or exception (the only two ways, other than the > >trap, to get into the kernel), and then you can multithread the whole > >damn thing, one subsystem at a time. > > Kind of how I had envisioned it... > > >You break the trap lock into inferior locks, one per subsystem (ie: VFS), > >and then break that lock further to push down a call at a time in the > >given subsystem. > > What exactly do you mean by breaking the lock down? Are you saying to > have a general lock cover a broad range of kernel access, then make a > more specific lock for a subset of that range and remove that subset > from the broader lock? Yes. The lock structure needs to be a hierarchy. Intention modes need to be implemented so that the hierarchy is not exclusive access all the way to the root lock (the kernel entry mutex). The hierarchy is necessary for deadlock avoidance, which is on the order of twice as efficient as deadlock detection, since you do not need to unwind state to recover from a conflict. 1) Start with a kernel entrancy lock in the exception, interrupt, and trap code. 2) Make the trap code reentrant. 3) Move the kernel entrancy locks from the trap code itself into the trap call targets (sysent[] functions). 4) Fan out the locking as a hierarchy so that sysent[] functions are classed by subsystem: VM, FS, Process, etc. 5) Convert the top level lock in the hierarchy to an IX lock so that synchronization is done on a subsystem basis instead of a trap entry basis. 6) Make one subsystem reentrant by taking the subsystem lock under the entrancy lock hierarchy, and fanning it out ona per-resource basis, locking common resources between calls in the subsystem. 7) Convert the per subsystem lock that is the parent for the common resource locks to an IX lock so that synchronization is done on a resource basis instead of a subsystem basis. 8) Start in on the next subsystem. Where possible, all new code should be single-entry, single-exit, so the lock state is clear, and is seperate from a direct understanding of the function of the code. Otherwise, it will end up being one or two people doing a massive code integration instead of a bunch of people doing little code integrations... you can't break the problem up into units if you have to have a holistic implementation (ie: all the code affected at once). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199606051800.LAA29231>