Date: Tue, 2 Jan 2001 12:03:19 -0800 (PST) From: Matthew Jacob <mjacob@feral.com> To: John Baldwin <jhb@FreeBSD.ORG> Cc: Doug Rabson <dfr@nlsystems.com>, alpha@FreeBSD.ORG Subject: RE: Interrupt threads Message-ID: <Pine.LNX.4.21.0101021150361.14503-100000@zeppo.feral.com> In-Reply-To: <XFMail.010102114301.jhb@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2 Jan 2001, John Baldwin wrote: > > On 28-Dec-00 Doug Rabson wrote: > > I started trying to work on changing the interrupt system so that the > > ithread runs with i/o interrupts disabled, which would allow us to remove > > the ugly code which tries to disable the interrupt sources. > > > > Unfortunately I immediately came up against a brick wall - the very first > > interrupt interrupted a proc which owned Giant and the ithread also wanted > > Giant. It tried to block, which switched back to the Giant owner with > > interrupts enabled, causing the interrupt to fire again. > > > > I'm not sure what the right approach to solving this is. Possibly we could > > 'lend' the mutex holder the IPL of the blocking thread in a similar way to > > priority propagation. Another idea is to link thread IPL directly to its > > priority so that when the ithread lends its priority, the mutex owner will > > automagically run with raised IPL. > > I detailed what I think is a way of using IPL to do this in a pair of e-mails > to Drew that Matt was copied on. I'll go dig them up and forward them to here. > However, tying IPL to a process or a mutex is a mistake I think. You can't tie > it to Giant because Giant is going away eventually. You can't tie it to a > process because the first context switch that doesn't switch into the ithread > will cause the interrupt to fire again. Instead, you basically have to futz > with the trapframe in the interrupt handler so it doesn't lower IPL upon > returning from teh interrupt, and don't lower the IPL again until the interrupt > handler completes and the ithread finishes. Interacting properly with ast() on > this makes it a bit more tricky. Let me go dig up my e-mails and forward them > to the list. Please do. I agree this is tricky. I'll reiterate what I said a while back (insofar as I recall) and add: 1. It should be a bug to have the scheduler switch to a process that returns to user space while leaving ithreads blocked. Hell, it should be a bug for any process that enters kernel context and acquires an SMP lock to not release it before returning to user space. 2. Giant is like any other mutex in this context- whoever holds Giant should be resumed if an ithread hits Giant. As soon as they drop Giant there should be an immediate cpu_switch which had better resume the [ one of the ] ithread[s] that had been blocked. 3. You should not have to do anything with a trapframe until you actually are returning from an interrupt. You can't assume that a 'return from interrupt' can return with IPL set to non-zero. This is such a POLA violation that I wouldn't expect it to work much of anywhere. Basically, you should take an interrupt and immediately have the ithread for it run (unless fast- and until we can get this right, only allow one non-fast interrupt at a time for alpha). If this ithread hits a lock and can't get it, the holder of that lock (whether Giant, Fred, Foo, Bar) gets resumed until it drops that lock, whereupon the ithread should be resumed. This can repeat until the ithread is done, whereupon a real return from interrupt can occur. Unfortunately, I think we've begun to experience what Solaris had to put up with in this respect, but so it goes. I would rather have a lot more in the above model but this strikes me as something which very closely adheres to what was the advertised SMP model we got from BSDi- and seems to have some commented out code about this. 4. What defeated me immediately upon looking at Doug's stuff is all of the thread pri seems to be a software construct, so has no real hardware linkage. Additionally, I did not take the time to try and find an easy way to find and reschdule the proc that holds a lock if I'm trying to acquire it from an ithread and am blocking. Seems to be a bit of work there. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-alpha" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.LNX.4.21.0101021150361.14503-100000>