From owner-freebsd-smp Thu Nov 16 14:54:34 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id E528937B4C5 for ; Thu, 16 Nov 2000 14:54:31 -0800 (PST) Received: from laptop.baldwin.cx (john@jhb-laptop.osd.bsdi.com [204.216.28.241]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id eAGMs9B70127; Thu, 16 Nov 2000 14:54:09 -0800 (PST) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20001116224259.DC1E6BA7A@io.yi.org> Date: Thu, 16 Nov 2000 14:54:48 -0800 (PST) From: John Baldwin To: Jake Burkholder Subject: Re: cvs commit: src/sys/kern kern_timeout.c Cc: jake@io.yi.org, cp@bsdi.com, smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 16-Nov-00 Jake Burkholder wrote: >> >> On 16-Nov-00 John Baldwin wrote: >> > jhb 2000/11/16 13:20:53 PST >> > >> > Modified files: >> > sys/kern kern_timeout.c >> > Log: >> > The recent changes to msleep() and mawait() resulted in timeout() and >> > untimeout() not being called with Giant in those functions. For now, >> > use the sched_lock to protect the callout wheel in softclock() and in >> > the various timeout and callout functions. >> > >> > Noticed by: tegge >> >> Big Kudos to Tor for finding this! Using the sched_lock here is more or >> less a >> bandaid for now. A possible long-term solution might be to use a dedicated >> sleep lock to protect the callout wheel. However, if you do this, then you >> can't be holding a spinlock when you call into timeout and friends. If we >> add >> in a lock to protect teh sleep queues, then we might be able to push >> sched_lock >> further down into msleep() and mawait(), so that we don't need sched_lock >> until >> after we call timeout and before we call untimeout, but I'm still thinking >> about this. Does anyone else have any ideas? One thing I am worried about >> is >> whether the overhead of the extra locks involved (sleepqueue locks, probably >> process locks (but we will need those anyway), and a callout wheel lock) >> will >> end up negating the performance gain from not having sched_lock protect so >> many >> different things. > > I think we need a separate spin lock for the callout wheel, ala BSD/OS's > callout_mtx. Hardclock looks at the callout wheel and is now a fast > interrupt, so it can't acquire a sleep mutex. Its a little paranoid > because hardclock doesn't actually traverse any lists, it just checks > if the current callout bucket is empty, and potentially schedules > softclock, but you could miss a very short timeout on an smp system. > ticks could also get incremented in the middle of softclock's test > for if the callout's time has come. > > I have patches that do this and make softclock INTR_MPSAFE, I just need > to test them. Ok. I was about to check the BSD/OS code to see how this was done there. > There's actually another major problem with this. The run queue and > sleep queue use the same list linkage in struct proc, so its not > safe to release sched_lock while you're on the sleep queue. If > the process blocks on giant in CURSIG, the sleep queue will get > corrupted. We really need to split the run queue/sleep queue > linkage. Ugh, ok. I'll do this next then. Grrrr. > Jake -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message