From owner-freebsd-smp Sun Oct 1 16:54:17 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 16BA237B503 for ; Sun, 1 Oct 2000 16:54:15 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001001235414.VYOZ26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Sun, 1 Oct 2000 16:54:14 -0700 Content-Length: 1710 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010010331.VAA14982@harmony.village.org> Date: Sun, 01 Oct 2000 16:54:16 -0700 (PDT) From: John Baldwin To: Warner Losh Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 01-Oct-00 Warner Losh wrote: > In message John Baldwin writes: >: Grr. I'd test it on my laptop, but pccard isnt' attaching to my >: pcic_pci controller. I've tried the following patch but no go: > > The reason is that the pcic_p.c isn't putting the cardbus bridge into > legacy mode correctly. I'd bet a case of beer that the patch you > included won't work. Ok. >: pcic-pci0: at device 4.0 on pci0 > > These are always a pita. what kind of laptop do you have? Dell Inspiron 5000e > sudo pciconf -l hostb0@pci0:0:0: class=0x060000 card=0x00000000 chip=0x71908086 rev=0x03 hdr=0x00 pcib1@pci0:1:0: class=0x060400 card=0x00000000 chip=0x71918086 rev=0x03 hdr=0x01 pcic-pci0@pci0:4:0: class=0x060700 card=0x00cc1028 chip=0xac1c104c rev=0x01 hdr=0x02 pcic-pci1@pci0:4:1: class=0x060700 card=0x00cc1028 chip=0xac1c104c rev=0x01 hdr=0x02 isab0@pci0:7:0: class=0x068000 card=0x00000000 chip=0x71108086 rev=0x02 hdr=0x00 atapci0@pci0:7:1: class=0x010180 card=0x00000000 chip=0x71118086 rev=0x01 hdr=0x00 uhci0@pci0:7:2: class=0x0c0300 card=0x00000000 chip=0x71128086 rev=0x01 hdr=0x00 none0@pci0:7:3: class=0x068000 card=0x00000000 chip=0x71138086 rev=0x03 hdr=0x00 pcm0@pci0:8:0: class=0x040100 card=0x00cc1028 chip=0x1978125d rev=0x10 hdr=0x00 none1@pci0:16:0: class=0x078000 card=0x20001668 chip=0x044811c1 rev=0x01 hdr=0x00 none2@pci1:0:0: class=0x030000 card=0x00cc1028 chip=0x4c461002 rev=0x02 hdr=0x00 > Warner -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 1 17: 0: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 0A7B437B66D; Sun, 1 Oct 2000 17:00:04 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id SAA01991; Sun, 1 Oct 2000 18:00:02 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id SAA03285; Sun, 1 Oct 2000 18:00:02 -0600 (MDT) Message-Id: <200010020000.SAA03285@harmony.village.org> To: John Baldwin Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org In-reply-to: Your message of "Sun, 01 Oct 2000 16:54:16 PDT." References: Date: Sun, 01 Oct 2000 18:00:02 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message John Baldwin writes: : Dell Inspiron 5000e Thanks for the info. Does the BIOS have a setting for legacy mode for the cardbus bridge? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 7:50:30 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sasami.jurai.net (sasami.jurai.net [63.67.141.99]) by hub.freebsd.org (Postfix) with ESMTP id 9E1BA37B502 for ; Mon, 2 Oct 2000 07:50:26 -0700 (PDT) Received: from localhost (winter@localhost) by sasami.jurai.net (8.9.3/8.8.7) with ESMTP id KAA55773; Mon, 2 Oct 2000 10:50:23 -0400 (EDT) Date: Mon, 2 Oct 2000 10:50:22 -0400 (EDT) From: "Matthew N. Dodd" To: Steve Kargl Cc: freebsd-smp@FreeBSD.ORG Subject: Re: 2 panics with MP kernels In-Reply-To: <200009282017.e8SKHPg00809@troutmask.apl.washington.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 28 Sep 2000, Steve Kargl wrote: > Panic 1 is related to havig "options USER_LDT" in the > kernel config file. Info I have so far is (transcribed by hand): Bingo! This is the panic that was causing me all sorts of problems. Looks like gd_currentldt is trashing gd_cpuid at some point. Take out 'options USER_LDT' and 'device random' and you should be able to get -CURRENT running. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | This Space For Rent | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 8:39:15 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 3D0C037B502 for ; Mon, 2 Oct 2000 08:39:12 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.0/8.11.0) id e92FiNw13072; Mon, 2 Oct 2000 08:44:23 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200010021544.e92FiNw13072@troutmask.apl.washington.edu> Subject: Re: 2 panics with MP kernels In-Reply-To: from "Matthew N. Dodd" at "Oct 2, 2000 10:50:22 am" To: "Matthew N. Dodd" Date: Mon, 2 Oct 2000 08:44:23 -0700 (PDT) Cc: freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Matthew N. Dodd wrote: > On Thu, 28 Sep 2000, Steve Kargl wrote: > > Panic 1 is related to havig "options USER_LDT" in the > > kernel config file. Info I have so far is (transcribed by hand): > > This is the panic that was causing me all sorts of problems. > > Looks like gd_currentldt is trashing gd_cpuid at some point. > > Take out 'options USER_LDT' and 'device random' and you should be able to > get -CURRENT running. > I can toss 'options USER_LDT', but isn't 'device random' now mandatory? The 20000710 entry in src/UPDATING suggests that crypto tools will break. -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 8:50:11 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sasami.jurai.net (sasami.jurai.net [63.67.141.99]) by hub.freebsd.org (Postfix) with ESMTP id 528C637B66C for ; Mon, 2 Oct 2000 08:50:06 -0700 (PDT) Received: from localhost (winter@localhost) by sasami.jurai.net (8.9.3/8.8.7) with ESMTP id LAA56679; Mon, 2 Oct 2000 11:50:00 -0400 (EDT) Date: Mon, 2 Oct 2000 11:50:00 -0400 (EDT) From: "Matthew N. Dodd" To: Steve Kargl Cc: freebsd-smp@FreeBSD.ORG Subject: Re: 2 panics with MP kernels In-Reply-To: <200010021544.e92FiNw13072@troutmask.apl.washington.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 2 Oct 2000, Steve Kargl wrote: > I can toss 'options USER_LDT', but isn't 'device random' now > mandatory? The 20000710 entry in src/UPDATING suggests that crypto > tools will break. Ah, well I wouldn't know since I don't bother with the system OpenSSL stuff. I'm a bit annoyed with IPX being broken by the lack of 'device random' but I'm told someone is working on it. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | This Space For Rent | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 9:37:30 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id 9154637B502 for ; Mon, 2 Oct 2000 09:37:24 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13g8aZ-000Fq3-00; Mon, 2 Oct 2000 18:37:15 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.0/8.11.0) with ESMTP id e92GbJj01791; Mon, 2 Oct 2000 18:37:19 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200010021637.e92GbJj01791@grimreaper.grondar.za> To: Steve Kargl Cc: freebsd-smp@FreeBSD.ORG Subject: Re: 2 panics with MP kernels References: <200010021544.e92FiNw13072@troutmask.apl.washington.edu> In-Reply-To: <200010021544.e92FiNw13072@troutmask.apl.washington.edu> ; from Steve Kargl "Mon, 02 Oct 2000 08:44:23 MST." Date: Mon, 02 Oct 2000 18:37:19 +0200 From: Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Take out 'options USER_LDT' and 'device random' and you should be able to > > get -CURRENT running. > > > > I can toss 'options USER_LDT', but isn't 'device random' > now mandatory? The 20000710 entry in src/UPDATING suggests > that crypto tools will break. The only unstableness about device random that I'm aware of is a guaranteed panic if you try to unload the module. If its built in, no problem. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 9:46:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 2285C37B66C for ; Mon, 2 Oct 2000 09:46:46 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.0/8.11.0) id e92Gpml13396; Mon, 2 Oct 2000 09:51:48 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200010021651.e92Gpml13396@troutmask.apl.washington.edu> Subject: Re: 2 panics with MP kernels In-Reply-To: <200010021637.e92GbJj01791@grimreaper.grondar.za> from Mark Murray at "Oct 2, 2000 06:37:19 pm" To: Mark Murray Date: Mon, 2 Oct 2000 09:51:48 -0700 (PDT) Cc: freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Mark Murray wrote: > > > Take out 'options USER_LDT' and 'device random' and you should be able to > > > get -CURRENT running. > > > > > > > I can toss 'options USER_LDT', but isn't 'device random' > > now mandatory? The 20000710 entry in src/UPDATING suggests > > that crypto tools will break. > > The only unstableness about device random that I'm aware of is a > guaranteed panic if you try to unload the module. If its built in, > no problem. > If you can't find my original email, the panic message is below my .sig. This is for a MP kernel from a few days ago. The panic is probably in the new mutex code, but /dev/random starts the trace in db. -- Steve panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ @/usr/src/sys/modules/randomdev/../../dev/randomdev/yarrow.c:515 cpuid = 1; lapic.id = 00000000 Debugger("panic') CPU1 stopping CPUs: 0x00000001 stopped Stopped at Debugger+0x38: movb $0,in_Debugger.684 db>trace Debugger(c0277da1) panic witness_enter _mtx_enter random_harvest_internal write_random random_write ufs_vnoperatespec vn_write dofilewrite write syscall2 Xint0x80_syscall To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 9:58:35 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id C296937B502 for ; Mon, 2 Oct 2000 09:58:29 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13g8v2-000G2D-00; Mon, 2 Oct 2000 18:58:25 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.0/8.11.0) with ESMTP id e92GwUj01890; Mon, 2 Oct 2000 18:58:30 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200010021658.e92GwUj01890@grimreaper.grondar.za> To: Steve Kargl Cc: Mark Murray , freebsd-smp@FreeBSD.ORG Subject: Re: 2 panics with MP kernels References: <200010021651.e92Gpml13396@troutmask.apl.washington.edu> In-Reply-To: <200010021651.e92Gpml13396@troutmask.apl.washington.edu> ; from Steve Kargl "Mon, 02 Oct 2000 09:51:48 MST." Date: Mon, 02 Oct 2000 18:58:30 +0200 From: Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Mark Murray wrote: > > > > Take out 'options USER_LDT' and 'device random' and you should be able to > > > > get -CURRENT running. > > > > > > > > > > I can toss 'options USER_LDT', but isn't 'device random' > > > now mandatory? The 20000710 entry in src/UPDATING suggests > > > that crypto tools will break. > > > > The only unstableness about device random that I'm aware of is a > > guaranteed panic if you try to unload the module. If its built in, > > no problem. > > > > If you can't find my original email, the panic message is > below my .sig. This is for a MP kernel from a few days > ago. The panic is probably in the new mutex code, but > /dev/random starts the trace in db. > > -- > Steve > > panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ > @/usr/src/sys/modules/randomdev/../../dev/randomdev/yarrow.c:515 > cpuid = 1; lapic.id = 00000000 > Debugger("panic') > > CPU1 stopping CPUs: 0x00000001 stopped > Stopped at Debugger+0x38: movb $0,in_Debugger.684 > > db>trace > Debugger(c0277da1) > panic > witness_enter > _mtx_enter > random_harvest_internal > write_random > random_write > ufs_vnoperatespec > vn_write > dofilewrite > write > syscall2 > Xint0x80_syscall > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 9:59:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id C0E3037B502 for ; Mon, 2 Oct 2000 09:59:46 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13g8wJ-000G3p-00; Mon, 2 Oct 2000 18:59:44 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.0/8.11.0) with ESMTP id e92Gxoj01904; Mon, 2 Oct 2000 18:59:50 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200010021659.e92Gxoj01904@grimreaper.grondar.za> To: Steve Kargl Cc: freebsd-smp@FreeBSD.ORG Subject: Re: 2 panics with MP kernels References: <200010021651.e92Gpml13396@troutmask.apl.washington.edu> In-Reply-To: <200010021651.e92Gpml13396@troutmask.apl.washington.edu> ; from Steve Kargl "Mon, 02 Oct 2000 09:51:48 MST." Date: Mon, 02 Oct 2000 18:59:50 +0200 From: Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org [ Sorry - I fat-fingered the other one ] > If you can't find my original email, the panic message is > below my .sig. This is for a MP kernel from a few days > ago. The panic is probably in the new mutex code, but > /dev/random starts the trace in db. I've never seen that actual one. I have seen similar mutex panics, though. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 16:26:52 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 2FBDB37B503 for ; Mon, 2 Oct 2000 16:26:50 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.1/8.11.1) id e92NWan00373 for freebsd-smp@freebsd.org; Mon, 2 Oct 2000 16:32:36 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200010022332.e92NWan00373@troutmask.apl.washington.edu> Subject: witness code status? To: freebsd-smp@freebsd.org Date: Mon, 2 Oct 2000 16:32:36 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org What's the current status of the integration of BSDi's witness code? If I include "options WITNESS" in my kernel config file, I get an immediate panic at boot. My sources are from today (01 Oct 00), and the system is currently running a MP kernel (built without the witness code). PS: Yes, I know there will be periods of instability, but you (the SMP developers) can't possibly have all combinations of MP hardware. PPS: "options USER_LDT" is currently broken for MP kernels. -- Steve panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ @/usr/src/sys/modules/randomdev/../../dev/randomdev/yarrow.c:515 cpuid = 1; lapic.id = 00000000 Debugger("panic') CPU1 stopping CPUs: 0x00000001 stopped Stopped at Debugger+0x38: movb $0,in_Debugger.684 db>trace Debugger(c0277da1) panic witness_enter _mtx_enter random_harvest_internal write_random random_write ufs_vnoperatespec vn_write dofilewrite write syscall2 Xint0x80_syscall To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 16:33: 8 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 1AAC137B503; Mon, 2 Oct 2000 16:33:03 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001002233302.IJNU26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Mon, 2 Oct 2000 16:33:02 -0700 Content-Length: 2567 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Mon, 02 Oct 2000 16:33:03 -0700 (PDT) From: John Baldwin To: Doug Rabson Subject: Re: Status update Cc: alpha@freebsd.org, cp@bsdi.com, smp@freebsd.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 26-Sep-00 Doug Rabson wrote: > On Tue, 26 Sep 2000, Doug Rabson wrote: > >> On Sat, 23 Sep 2000, John Baldwin wrote: >> > >> > There are also a few more weirdism's on the alpha. In a few places in >> > sys/kern, we call spl0() instead of splx(). I've added some debugging >> > code to >> > do a printf() if we aren't actually at IPL_0 (what spl0 used to do) after >> > the >> > mtx_exit(). It does trigger in several cases during /etc/rc at least, but >> > the >> > machine seems to be running stable regardless (I'll be running a >> > buildworld -j >> > 8 tonight to stress test it). My question is: is it ok for the code to >> > run >> > with some interrupts disabled or do we need to replace the calls to spl0() >> > with enable_intr()? >> >> I'm testing this now and I'm seeing a flood of diagnostic messages like: >> >> ../../kern/kern_fork.c:537:fork1() spl0 needs fixing >> >> I think these are all due to the fact that sched_lock is held at that >> point. The preceding mtx_exit() just decrements the recursion count. >> I haven't worked out exactly why sched_lock is held here because it >> shouldn't be. Possibly its because we don't clear pcb_schednest in >> cpu_fork(). > > As I suspected, clearing pcb_schednest makes this lot go away. Try this > version of vm_machdep.c: > > Index: vm_machdep.c > =================================================================== > RCS file: /home/ncvs/src/sys/alpha/alpha/vm_machdep.c,v > retrieving revision 1.33 > diff -u -r1.33 vm_machdep.c > --- vm_machdep.c 2000/09/07 01:32:39 1.33 > +++ vm_machdep.c 2000/09/26 08:38:55 > @@ -210,6 +210,13 @@ > up->u_pcb.pcb_context[2] = (u_long) p2; /* s2: a0 */ > up->u_pcb.pcb_context[7] = > (u_int64_t)switch_trampoline; /* ra: assembly magic */ > + > + /* > + * Clear the saved recursion count for sched_lock > + * since the child needs only one count which is > + * released in switch_trampoline. > + */ > + up->u_pcb.pcb_schednest = 0; > } > } The x86 code does this, so the alpha code should as well. Go ahead and commit this, it shouldnt' break anything. :-P I'll remove the extra cruft from the MTX_EXIT() assembly in mutex.h afterwards. > -- > Doug Rabson Mail: dfr@nlsystems.com -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 16:42: 3 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 6478C37B66D for ; Mon, 2 Oct 2000 16:41:59 -0700 (PDT) Received: (qmail 99851 invoked by uid 1142); 2 Oct 2000 23:41:57 -0000 Date: 2 Oct 2000 16:41:57 -0700 Date: Mon, 2 Oct 2000 16:41:39 -0700 From: Jason Evans To: Steve Kargl Cc: freebsd-smp@freebsd.org Subject: Re: witness code status? Message-ID: <20001002164139.I58256@canonware.com> References: <200010022332.e92NWan00373@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200010022332.e92NWan00373@troutmask.apl.washington.edu>; from sgk@troutmask.apl.washington.edu on Mon, Oct 02, 2000 at 04:32:36PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Oct 02, 2000 at 04:32:36PM -0700, Steve Kargl wrote: > What's the current status of the integration of BSDi's > witness code? If I include "options WITNESS" in my kernel > config file, I get an immediate panic at boot. My sources > are from today (01 Oct 00), and the system is currently > running a MP kernel (built without the witness code). > > PS: Yes, I know there will be periods of instability, but > you (the SMP developers) can't possibly have all combinations > of MP hardware. I (and others) have been running kernels with the witness code enabled for over a month. The witness code's purpose is to detect lock cycles that could cause deadlock, among other incorrect mutex uses. The stack trace and panic message in your email seem indicate a programming error. In other words, the witness code is doing its job. Jason > panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ > @/usr/src/sys/modules/randomdev/../../dev/randomdev/yarrow.c:515 > cpuid = 1; lapic.id = 00000000 > Debugger("panic') > > CPU1 stopping CPUs: 0x00000001 stopped > Stopped at Debugger+0x38: movb $0,in_Debugger.684 > > db>trace > Debugger(c0277da1) > panic > witness_enter > _mtx_enter > random_harvest_internal > write_random > random_write > ufs_vnoperatespec > vn_write > dofilewrite > write > syscall2 > Xint0x80_syscall To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 17: 6:26 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 5D2D737B503 for ; Mon, 2 Oct 2000 17:06:24 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.1/8.11.1) id e930Be940283; Mon, 2 Oct 2000 17:11:40 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200010030011.e930Be940283@troutmask.apl.washington.edu> Subject: Re: witness code status? In-Reply-To: <20001002164139.I58256@canonware.com> from Jason Evans at "Oct 2, 2000 04:41:39 pm" To: Jason Evans Date: Mon, 2 Oct 2000 17:11:40 -0700 (PDT) Cc: freebsd-smp@freebsd.org X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Jason Evans wrote: > On Mon, Oct 02, 2000 at 04:32:36PM -0700, Steve Kargl wrote: > > What's the current status of the integration of BSDi's > > witness code? If I include "options WITNESS" in my kernel > > config file, I get an immediate panic at boot. My sources > > are from today (01 Oct 00), and the system is currently > > running a MP kernel (built without the witness code). > > > > I (and others) have been running kernels with the witness code enabled for > over a month. The witness code's purpose is to detect lock cycles that > could cause deadlock, among other incorrect mutex uses. The stack trace > and panic message in your email seem indicate a programming error. In > other words, the witness code is doing its job. > Shoot, I was afraid you's say the above. I haven't been able to convince this machine to give me a crash dump for further analysis. I'll keep trying because this repeatable here. The KDEBUG option isn't documented in NOTES, but I noticed it strewn through kern_mutex.c. Will enabling this option provide more information that can help track this done? -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 17:21:11 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 9021C37B503 for ; Mon, 2 Oct 2000 17:21:09 -0700 (PDT) Received: (qmail 1347 invoked by uid 1142); 3 Oct 2000 00:21:05 -0000 Date: 2 Oct 2000 17:21:05 -0700 Date: Mon, 2 Oct 2000 17:20:48 -0700 From: Jason Evans To: Steve Kargl Cc: freebsd-smp@freebsd.org Subject: Re: witness code status? Message-ID: <20001002172048.J58256@canonware.com> References: <20001002164139.I58256@canonware.com> <200010030011.e930Be940283@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200010030011.e930Be940283@troutmask.apl.washington.edu>; from sgk@troutmask.apl.washington.edu on Mon, Oct 02, 2000 at 05:11:40PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Oct 02, 2000 at 05:11:40PM -0700, Steve Kargl wrote: > > Shoot, I was afraid you's say the above. I haven't been able to > convince this machine to give me a crash dump for further analysis. > I'll keep trying because this repeatable here. > > The KDEBUG option isn't documented in NOTES, but I noticed it strewn > through kern_mutex.c. Will enabling this option provide more > information that can help track this done? Hmm, I don't know what KDEBUG is for. However, you should definitely define the SMP_DEBUG option if you haven't already. I've been building kernels for a while now with the following additional pertinent options: options INVARIANTS options INVARIANT_SUPPORT options DIAGNOSTIC options SMP_DEBUG options WITNESS As for not being able to get a crash dump, I've been using ddb/gdb, so I'm not of much help there. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 17:32:14 2000 Delivered-To: freebsd-smp@freebsd.org Received: from tinker.exit.com (tinker.exit.com [206.223.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 425FE37B503 for ; Mon, 2 Oct 2000 17:32:00 -0700 (PDT) Received: from realtime.exit.com (realtime [206.223.0.5]) by tinker.exit.com (8.11.0/8.11.0) with ESMTP id e930VtL37606; Mon, 2 Oct 2000 17:31:55 -0700 (PDT) (envelope-from frank@exit.com) Received: (from frank@localhost) by realtime.exit.com (8.11.0/8.11.0) id e930Wba82841; Mon, 2 Oct 2000 17:32:37 -0700 (PDT) (envelope-from frank) From: Frank Mayhar Message-Id: <200010030032.e930Wba82841@realtime.exit.com> Subject: Re: witness code status? In-Reply-To: <20001002172048.J58256@canonware.com> from Jason Evans at "Oct 2, 2000 05:20:48 pm" To: Jason Evans Date: Mon, 2 Oct 2000 17:32:37 -0700 (PDT) Cc: Steve Kargl , freebsd-smp@FreeBSD.ORG Reply-To: frank@exit.com Organization: Exit Consulting X-Copyright0: Copyright 2000 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Jason Evans wrote: > Hmm, I don't know what KDEBUG is for. However, you should definitely It's the BSD/OS kernel debugger. Dunno why it's all over kern_mutex.c, unless that file was lifted bodily from BSD/OS. -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://store.exit.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 17:38:45 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 6F02A37B503 for ; Mon, 2 Oct 2000 17:38:43 -0700 (PDT) Received: from berserker.bsdi.com (cp@LOCALHOST [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id SAA04555; Mon, 2 Oct 2000 18:38:24 -0600 (MDT) Message-Id: <200010030038.SAA04555@berserker.bsdi.com> To: frank@exit.com Cc: Jason Evans , Steve Kargl , freebsd-smp@freebsd.org Subject: Re: witness code status? In-reply-to: Your message of "Mon, 02 Oct 2000 17:32:37 PDT." <200010030032.e930Wba82841@realtime.exit.com> From: Chuck Paterson Date: Mon, 02 Oct 2000 18:38:24 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org It was Chuck Frank Mayhar wrote on: Mon, 02 Oct 2000 17:32:37 PDT }Jason Evans wrote: }> Hmm, I don't know what KDEBUG is for. However, you should definitely } }It's the BSD/OS kernel debugger. Dunno why it's all over kern_mutex.c, }unless that file was lifted bodily from BSD/OS. }-- }Frank Mayhar frank@exit.com http://www.exit.com/ }Exit Consulting http://store.exit.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 20:41:53 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 6980837B503 for ; Mon, 2 Oct 2000 20:41:50 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001003034149.LJEG26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Mon, 2 Oct 2000 20:41:49 -0700 Content-Length: 1063 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010022332.e92NWan00373@troutmask.apl.washington.edu> Date: Mon, 02 Oct 2000 20:41:52 -0700 (PDT) From: John Baldwin To: Steve Kargl Subject: RE: witness code status? Cc: freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 02-Oct-00 Steve Kargl wrote: > What's the current status of the integration of BSDi's > witness code? If I include "options WITNESS" in my kernel > config file, I get an immediate panic at boot. My sources > are from today (01 Oct 00), and the system is currently > running a MP kernel (built without the witness code). I have never seen this panic, and I have used WITNESS in my kernels since I was first working on ithreads with Greg and Jake months ago. > PS: Yes, I know there will be periods of instability, but > you (the SMP developers) can't possibly have all combinations > of MP hardware. Well, I can still reliably lock up the quad xeon we have for test hardware @ BSDi. :-( > PPS: "options USER_LDT" is currently broken for MP kernels. Can you confirm that it works ok for UP? That would _greatly_ help to track this down if so. > -- > Steve -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Oct 2 21: 4:26 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 3DB1B37B503 for ; Mon, 2 Oct 2000 21:04:24 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001003040423.LPDG26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Mon, 2 Oct 2000 21:04:23 -0700 Content-Length: 1852 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20001002164139.I58256@canonware.com> Date: Mon, 02 Oct 2000 21:04:27 -0700 (PDT) From: John Baldwin To: Jason Evans Subject: Re: witness code status? Cc: freebsd-smp@FreeBSD.ORG, Steve Kargl Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 02-Oct-00 Jason Evans wrote: > On Mon, Oct 02, 2000 at 04:32:36PM -0700, Steve Kargl wrote: >> What's the current status of the integration of BSDi's >> witness code? If I include "options WITNESS" in my kernel >> config file, I get an immediate panic at boot. My sources >> are from today (01 Oct 00), and the system is currently >> running a MP kernel (built without the witness code). >> >> PS: Yes, I know there will be periods of instability, but >> you (the SMP developers) can't possibly have all combinations >> of MP hardware. > > I (and others) have been running kernels with the witness code enabled for > over a month. The witness code's purpose is to detect lock cycles that > could cause deadlock, among other incorrect mutex uses. The stack trace > and panic message in your email seem indicate a programming error. In > other words, the witness code is doing its job. Except that it's not in this case. The error indicates that mtx_enter() is being called with MTX_SPIN, but if you look in sys/dev/randomdev/*, all of the mtx_*() functions use MTX_DEF. It's a bug alright, but not in the random code. > Jason > >> panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ >> @/usr/src/sys/modules/randomdev/../../dev/randomdev/yarrow.c:515 >> cpuid = 1; lapic.id = 00000000 >> Debugger("panic') >> >> CPU1 stopping CPUs: 0x00000001 stopped >> Stopped at Debugger+0x38: movb $0,in_Debugger.684 >> >> db>trace >> Debugger(c0277da1) >> panic >> witness_enter >> _mtx_enter >> random_harvest_internal >> write_random >> random_write >> ufs_vnoperatespec >> vn_write >> dofilewrite >> write >> syscall2 >> Xint0x80_syscall -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 2:13:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from lists01.iafrica.com (lists01.iafrica.com [196.7.0.141]) by hub.freebsd.org (Postfix) with ESMTP id 7A38637B66C for ; Tue, 3 Oct 2000 02:13:37 -0700 (PDT) Received: from nwl.fw.uunet.co.za ([196.31.2.162]) by lists01.iafrica.com with esmtp (Exim 3.12 #2) id 13gO8G-0002RO-00; Tue, 03 Oct 2000 11:13:04 +0200 Received: (from nobody@localhost) by nwl.fw.uunet.co.za (8.8.8/8.6.9) id LAA25638; Tue, 3 Oct 2000 11:13:07 +0200 (SAST) Received: by nwl.fw.uunet.co.za via recvmail id 25455; Tue Oct 3 11:12:18 2000 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.0/8.11.0) with ESMTP id e939CDL06411; Tue, 3 Oct 2000 11:12:15 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200010030912.e939CDL06411@grimreaper.grondar.za> To: Jason Evans Cc: freebsd-smp@FreeBSD.ORG Subject: Re: witness code status? References: <20001002164139.I58256@canonware.com> In-Reply-To: <20001002164139.I58256@canonware.com> ; from Jason Evans "Mon, 02 Oct 2000 16:41:39 MST." Date: Tue, 03 Oct 2000 11:12:11 +0200 From: Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I (and others) have been running kernels with the witness code enabled for > over a month. The witness code's purpose is to detect lock cycles that > could cause deadlock, among other incorrect mutex uses. The stack trace > and panic message in your email seem indicate a programming error. In > other words, the witness code is doing its job. Jason, I'd like to ask your help on this; It seems that turning WITNESS on causes the /dev/random driver to screw up royally. I thought I was quite careful about that. Now, I'm a bit out of my depth. Could you give me a hand? M > > Jason > > > panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ > > @/usr/src/sys/modules/randomdev/../../dev/randomdev/yarrow.c:515 > > cpuid = 1; lapic.id = 00000000 > Debugger("panic') > > > > CPU1 stopping CPUs: 0x00000001 stopped > > Stopped at Debugger+0x38: movb $0,in_Debugger.684 > > > > db>trace > > Debugger(c0277da1) > > panic > > witness_enter > > _mtx_enter > > random_harvest_internal > > write_random > > random_write > > ufs_vnoperatespec > > vn_write > > dofilewrite > > write > > syscall2 > > Xint0x80_syscall > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 2:23:10 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-202-176-106.dsl.snfc21.pacbell.net [63.202.176.106]) by hub.freebsd.org (Postfix) with ESMTP id 0B87537B502 for ; Tue, 3 Oct 2000 02:23:08 -0700 (PDT) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e939OOh05799; Tue, 3 Oct 2000 02:24:26 -0700 (PDT) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200010030924.e939OOh05799@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: Mark Murray Cc: Jason Evans , freebsd-smp@FreeBSD.ORG Subject: Re: witness code status? In-reply-to: Your message of "Tue, 03 Oct 2000 11:12:11 +0200." <200010030912.e939CDL06411@grimreaper.grondar.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Oct 2000 02:24:24 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > I (and others) have been running kernels with the witness code enabled for > > over a month. The witness code's purpose is to detect lock cycles that > > could cause deadlock, among other incorrect mutex uses. The stack trace > > and panic message in your email seem indicate a programming error. In > > other words, the witness code is doing its job. > > Jason, > > I'd like to ask your help on this; It seems that turning WITNESS on causes > the /dev/random driver to screw up royally. > > I thought I was quite careful about that. Now, I'm a bit out of my depth. > > Could you give me a hand? ... > > > panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ You initialised the 'random harvest' mutex with MTX_SPIN, but you are trying to acquire it with MTX_DEF. You probably don't want a spin mutex for this anyway. -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 9:40:14 2000 Delivered-To: freebsd-smp@freebsd.org Received: from myrile.madriver.k12.oh.us (myrile.madriver.k12.oh.us [156.63.175.142]) by hub.freebsd.org (Postfix) with ESMTP id C142237B502 for ; Tue, 3 Oct 2000 09:40:10 -0700 (PDT) Received: (from mwhite@localhost) by myrile.madriver.k12.oh.us (8.9.3/8.9.3) id MAA11425 for freebsd-smp@freebsd.org; Tue, 3 Oct 2000 12:40:02 -0400 (EDT) (envelope-from mwhite) From: Matt White Message-Id: <200010031640.MAA11425@myrile.madriver.k12.oh.us> Subject: ServerWorks ServerSet III HE Chipset To: freebsd-smp@freebsd.org Date: Tue, 3 Oct 2000 12:40:02 -0400 (EDT) X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org FreeBSD SMP List; I'm looking to build a new FreeBSD 4.1 server to replace our 3.5 server that's getting beat to death every day. (And I must say, it responds to the beating quite well!) Right now I'm looking at possible boards and chipsets to use for a new server, and I'm wondering if anyone has any experence, good or bad, with the ServerWorks chipsets. I've searched the mailing list archives and can't seem to find an answer. (But I did notice that this question comes up every now and then!) In particular I'm looking at the SuperMicro 370DL3 or 370DE6 boards. One has the HE chipset, the other has the LE chipset. Any comments, good or bad? Thanks, - Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 9:53:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from io.yi.org (24.67.218.234.bc.wave.home.com [24.67.218.234]) by hub.freebsd.org (Postfix) with ESMTP id 16AFC37B502 for ; Tue, 3 Oct 2000 09:53:45 -0700 (PDT) Received: from io.yi.org (localhost.gvcl1.bc.wave.home.com [127.0.0.1]) by io.yi.org (Postfix) with ESMTP id 56138BA77; Tue, 3 Oct 2000 09:53:44 -0700 (PDT) X-Mailer: exmh version 2.1.1 10/15/1999 To: Steve Kargl Cc: freebsd-smp@FreeBSD.ORG Subject: Re: witness code status? In-Reply-To: Message from Steve Kargl of "Mon, 02 Oct 2000 16:32:36 PDT." <200010022332.e92NWan00373@troutmask.apl.washington.edu> Mime-Version: 1.0 Content-Type: multipart/mixed ; boundary="==_Exmh_10802101420" Date: Tue, 03 Oct 2000 09:53:44 -0700 From: Jake Burkholder Message-Id: <20001003165344.56138BA77@io.yi.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This is a multipart MIME message. --==_Exmh_10802101420 Content-Type: text/plain; charset=us-ascii > What's the current status of the integration of BSDi's > witness code? If I include "options WITNESS" in my kernel > config file, I get an immediate panic at boot. My sources > are from today (01 Oct 00), and the system is currently > running a MP kernel (built without the witness code). This makes no sense to me; I can't figure out why the panic is happening at all. Can you send me your kernel config file? Also, please make absolutely sure that your sources are up to date and build a new world and kernel. What's happening is the w_spin flag in the witness structure for Mark's harvest mutex is being set when it should not be. But I can't see where this could happen unless someone is randomly overwriting memory. > > PS: Yes, I know there will be periods of instability, but > you (the SMP developers) can't possibly have all combinations > of MP hardware. > > PPS: "options USER_LDT" is currently broken for MP kernels. > Attached is a patch that should fix this. Please let me know and I'll commit it. (Works here.) --==_Exmh_10802101420 Content-Type: text/plain ; name="idle.diff"; charset=us-ascii Content-Description: idle.diff Content-Disposition: attachment; filename="idle.diff" Index: kern_idle.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_idle.c,v retrieving revision 1.5 diff -u -r1.5 kern_idle.c --- kern_idle.c 2000/09/22 03:19:24 1.5 +++ kern_idle.c 2000/10/03 16:44:22 @@ -4,6 +4,9 @@ * $FreeBSD: src/sys/kern/kern_idle.c,v 1.5 2000/09/22 03:19:24 msmith Exp $ */ +#ifdef __i386__ +#include "opt_user_ldt.h" +#endif #include "opt_ktrace.h" #include --==_Exmh_10802101420-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 10:35: 9 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id F130737B503 for ; Tue, 3 Oct 2000 10:35:05 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.1/8.11.1) id e93Heqq14039; Tue, 3 Oct 2000 10:40:52 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200010031740.e93Heqq14039@troutmask.apl.washington.edu> Subject: Re: witness code status? In-Reply-To: <20001003165344.56138BA77@io.yi.org> from Jake Burkholder at "Oct 3, 2000 09:53:44 am" To: Jake Burkholder Date: Tue, 3 Oct 2000 10:40:52 -0700 (PDT) Cc: freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Jake Burkholder wrote: > > What's the current status of the integration of BSDi's > > witness code? If I include "options WITNESS" in my kernel > > config file, I get an immediate panic at boot. My sources > > are from today (01 Oct 00), and the system is currently > > running a MP kernel (built without the witness code). > > This makes no sense to me; I can't figure out why the panic > is happening at all. Can you send me your kernel config file? It's attached. I've also put some info up on the web at http://troutmask.apl.washington/edu/~kargl/freebsd/TROUTMASK http://troutmask.apl.washington/edu/~kargl/freebsd/dmesg-pre.SMPng http://troutmask.apl.washington/edu/~kargl/freebsd/dmesg-post.SMPng http://troutmask.apl.washington/edu/~kargl/freebsd/sysctl.data http://troutmask.apl.washington/edu/~kargl/freebsd/mptable.out > Also, please make absolutely sure that your sources are up to > date and build a new world and kernel. I pull my sources from cvsup7.freebsd.org, and unless something significant happened after the timestamp on /usr/sup/src-all: -rw-r--r-- 1 root wheel - 4794104 Oct 2 08:56 checkouts.cvs:. then I should be fairly up to date. > > PPS: "options USER_LDT" is currently broken for MP kernels. > > > > Attached is a patch that should fix this. > Please let me know and I'll commit it. (Works here.) I'm building a kernel, now. -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 10:42:17 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 88C9B37B502; Tue, 3 Oct 2000 10:42:14 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.1/8.11.1) id e93Hm1t16585; Tue, 3 Oct 2000 10:48:01 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200010031748.e93Hm1t16585@troutmask.apl.washington.edu> Subject: Re: witness code status? In-Reply-To: from John Baldwin at "Oct 2, 2000 08:41:52 pm" To: John Baldwin Date: Tue, 3 Oct 2000 10:48:01 -0700 (PDT) Cc: freebsd-smp@FreeBSD.org X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org John Baldwin wrote: > > > PPS: "options USER_LDT" is currently broken for MP kernels. > > Can you confirm that it works ok for UP? That would _greatly_ help > to track this down if so. > A UP kernel works fine with "options USER_LDT" (and coincidentally device random). "options USER_LDT" troutmask:kargl[201] ll /boot/kernel.sgk/kernel -r-xr-xr-x 1 root wheel schg 2158664 Sep 26 10:33 /boot/kernel.sgk/kernel troutmask:kargl[202] nm /boot/kernel.sgk/kernel | grep ldt c02d5e08 B _default_ldt c02b1530 D currentldt c02b1530 D gd_currentldt c024f2ac t i386_get_ldt c024f388 t i386_set_ldt c02d5d40 B ldt c02b1ae0 d ldt_segs c024eb6c T lldt c024f138 T set_user_ldt c024f170 T user_ldt_alloc c024f248 T user_ldt_free troutmask:kargl[203] -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 11:25: 2 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 62F6B37B66D for ; Tue, 3 Oct 2000 11:24:57 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001003182456.SWJW26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Tue, 3 Oct 2000 11:24:56 -0700 Content-Length: 1335 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="_=XFMail.1.4.0.FreeBSD:001003112310:3911=_" In-Reply-To: <20001003165344.56138BA77@io.yi.org> Date: Tue, 03 Oct 2000 11:25:00 -0700 (PDT) From: John Baldwin To: Jake Burkholder Subject: Re: witness code status? Cc: freebsd-smp@FreeBSD.ORG, Steve Kargl Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This message is in MIME format --_=XFMail.1.4.0.FreeBSD:001003112310:3911=_ Content-Type: text/plain; charset=us-ascii On 03-Oct-00 Jake Burkholder wrote: > Attached is a patch that should fix this. > Please let me know and I'll commit it. (Works here.) Ugh. How and why does this work exactly, and should we be #include'ing opt_user_ldt.h in an include file instead, or should we move USER_LDT to opt_global.h perhaps? -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ --_=XFMail.1.4.0.FreeBSD:001003112310:3911=_ Content-Type: text/plain ; name="idle.diff"; charset=us-ascii Content-Description: idle.diff Content-Disposition: attachment; filename="idle.diff" Index: kern_idle.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_idle.c,v retrieving revision 1.5 diff -u -r1.5 kern_idle.c --- kern_idle.c 2000/09/22 03:19:24 1.5 +++ kern_idle.c 2000/10/03 16:44:22 @@ -4,6 +4,9 @@ * $FreeBSD: src/sys/kern/kern_idle.c,v 1.5 2000/09/22 03:19:24 msmith Exp $ */ +#ifdef __i386__ +#include "opt_user_ldt.h" +#endif #include "opt_ktrace.h" #include --_=XFMail.1.4.0.FreeBSD:001003112310:3911=_-- End of MIME message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 11:25: 6 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 3577C37B66F; Tue, 3 Oct 2000 11:24:59 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001003182458.SWKB26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Tue, 3 Oct 2000 11:24:58 -0700 Content-Length: 1567 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010030924.e939OOh05799@mass.osd.bsdi.com> Date: Tue, 03 Oct 2000 11:25:02 -0700 (PDT) From: John Baldwin To: Mike Smith Subject: Re: witness code status? Cc: freebsd-smp@FreeBSD.ORG, Jason Evans , Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 03-Oct-00 Mike Smith wrote: >> > I (and others) have been running kernels with the witness code enabled for >> > over a month. The witness code's purpose is to detect lock cycles that >> > could cause deadlock, among other incorrect mutex uses. The stack trace >> > and panic message in your email seem indicate a programming error. In >> > other words, the witness code is doing its job. >> >> Jason, >> >> I'd like to ask your help on this; It seems that turning WITNESS on causes >> the /dev/random driver to screw up royally. >> >> I thought I was quite careful about that. Now, I'm a bit out of my depth. >> >> Could you give me a hand? > ... >> > > panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ > > You initialised the 'random harvest' mutex with MTX_SPIN, but you are > trying to acquire it with MTX_DEF. You probably don't want a spin mutex > for this anyway. Uh. What code are you looking at Mike? [11:23:47] (ttyp0) john@laptop:/sys/dev/randomdev > grep mtx_init * yarrow.c: mtx_init(&random_reseed_mtx, "random reseed", MTX_DEF); yarrow.c: mtx_init(&random_harvest_mtx, "random harvest", MTX_DEF); This is _not_ a bug in the random code. I have used device random with the random thread and WITNESS w/o any problems at work. I'll give it a go here in just a sec. But this is not Mark's fault. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 12: 2:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id BA42137B66F; Tue, 3 Oct 2000 12:02:40 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.1/8.11.1) id e93J8OS00830; Tue, 3 Oct 2000 12:08:24 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200010031908.e93J8OS00830@troutmask.apl.washington.edu> Subject: Re: witness code status? In-Reply-To: from John Baldwin at "Oct 3, 2000 11:25:00 am" To: John Baldwin Date: Tue, 3 Oct 2000 12:08:23 -0700 (PDT) Cc: Jake Burkholder , freebsd-smp@FreeBSD.org X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org John Baldwin wrote: > > On 03-Oct-00 Jake Burkholder wrote: > > Attached is a patch that should fix this. > > Please let me know and I'll commit it. (Works here.) > > Ugh. How and why does this work exactly, and should we be > #include'ing opt_user_ldt.h in an include file instead, or > should we move USER_LDT to opt_global.h perhaps? > Well, it appears to work because troutmask:sgk[247] find /sys/ -name \*.h | xargs grep USER_LDT /sys/compile/TROUTMASK/opt_user_ldt.h:#define USER_LDT 1 /sys/i386/include/globaldata.h:#ifdef USER_LDT /sys/i386/include/globals.h:#ifdef USER_LDT /sys/i386/include/globals.h:#ifdef USER_LDT And kern_idle.c includes #include #include which suggests that yes USER_LDT should move to opt_global.h. -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 3 12:19: 6 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 5841A37B66C for ; Tue, 3 Oct 2000 12:19:04 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001003191903.TLXF26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Tue, 3 Oct 2000 12:19:03 -0700 Content-Length: 1182 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010031908.e93J8OS00830@troutmask.apl.washington.edu> Date: Tue, 03 Oct 2000 12:19:07 -0700 (PDT) From: John Baldwin To: Steve Kargl Subject: Re: witness code status? Cc: freebsd-smp@FreeBSD.org, Jake Burkholder Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 03-Oct-00 Steve Kargl wrote: > John Baldwin wrote: >> >> On 03-Oct-00 Jake Burkholder wrote: >> > Attached is a patch that should fix this. >> > Please let me know and I'll commit it. (Works here.) >> >> Ugh. How and why does this work exactly, and should we be >> #include'ing opt_user_ldt.h in an include file instead, or >> should we move USER_LDT to opt_global.h perhaps? >> > > Well, it appears to work because > troutmask:sgk[247] find /sys/ -name \*.h | xargs grep USER_LDT > /sys/compile/TROUTMASK/opt_user_ldt.h:#define USER_LDT 1 > /sys/i386/include/globaldata.h:#ifdef USER_LDT > /sys/i386/include/globals.h:#ifdef USER_LDT > /sys/i386/include/globals.h:#ifdef USER_LDT Argh. Ok. In some other header files we create dummy variables so that the structs remain the same. > And kern_idle.c includes > >#include >#include > > which suggests that yes USER_LDT should move to opt_global.h. I'll work up a patch to do this in a sec. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 4 0: 0: 8 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id A192737B503 for ; Tue, 3 Oct 2000 23:59:57 -0700 (PDT) Received: (qmail 12263 invoked from network); 4 Oct 2000 06:59:53 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 4 Oct 2000 06:59:53 -0000 Date: Wed, 4 Oct 2000 17:59:49 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: John Baldwin Cc: Steve Kargl , freebsd-smp@FreeBSD.ORG, Jake Burkholder Subject: Re: witness code status? In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 3 Oct 2000, John Baldwin wrote: > On 03-Oct-00 Steve Kargl wrote: > > John Baldwin wrote: > >> > >> On 03-Oct-00 Jake Burkholder wrote: > >> > Attached is a patch that should fix this. > >> > Please let me know and I'll commit it. (Works here.) > >> > >> Ugh. How and why does this work exactly, and should we be > >> #include'ing opt_user_ldt.h in an include file instead, or > >> should we move USER_LDT to opt_global.h perhaps? > >> > > > > Well, it appears to work because > > troutmask:sgk[247] find /sys/ -name \*.h | xargs grep USER_LDT > > /sys/compile/TROUTMASK/opt_user_ldt.h:#define USER_LDT 1 > > /sys/i386/include/globaldata.h:#ifdef USER_LDT > > /sys/i386/include/globals.h:#ifdef USER_LDT > > /sys/i386/include/globals.h:#ifdef USER_LDT > > Argh. Ok. In some other header files we create dummy variables > so that the structs remain the same. > > > And kern_idle.c includes > > > >#include > >#include > > > > which suggests that yes USER_LDT should move to opt_global.h. No, it suggests another broken header and/or an #include mess. MI code cannot reference MD variables (global or otherwise) directly. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 2:39:36 2000 Delivered-To: freebsd-smp@freebsd.org Received: from netplex.com.au (adsl-63-207-30-186.dsl.snfc21.pacbell.net [63.207.30.186]) by hub.freebsd.org (Postfix) with ESMTP id 2527D37B502 for ; Thu, 5 Oct 2000 02:39:34 -0700 (PDT) Received: from netplex.com.au (peter@localhost [127.0.0.1]) by netplex.com.au (8.11.0/8.9.3) with ESMTP id e959cIH25740; Thu, 5 Oct 2000 02:38:18 -0700 (PDT) (envelope-from peter@netplex.com.au) Message-Id: <200010050938.e959cIH25740@netplex.com.au> X-Mailer: exmh version 2.1.1 10/15/1999 To: Matt White Cc: freebsd-smp@FreeBSD.ORG Subject: Re: ServerWorks ServerSet III HE Chipset In-Reply-To: <200010031640.MAA11425@myrile.madriver.k12.oh.us> Date: Thu, 05 Oct 2000 02:38:18 -0700 From: Peter Wemm Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Matt White wrote: > FreeBSD SMP List; > > I'm looking to build a new FreeBSD 4.1 server to replace our 3.5 server > that's getting beat to death every day. (And I must say, it responds to > the beating quite well!) > > Right now I'm looking at possible boards and chipsets to use for a new > server, and I'm wondering if anyone has any experence, good or bad, with > the ServerWorks chipsets. I've searched the mailing list archives and > can't seem to find an answer. (But I did notice that this question comes > up every now and then!) > > In particular I'm looking at the SuperMicro 370DL3 or 370DE6 boards. One > has the HE chipset, the other has the LE chipset. > > Any comments, good or bad? I cannot comment on the supermicro, but I have a Tyan Thunder 2500 which is based on the RCC HE chipset. It is a very sweet board, and goes like the proverbial bat out of hell. I believe Paul Saab had one of them with 2x733MHz cpus beating a 4x550MHz Xeon system at buildworld time from MFS ramdisks. The Quad Xeons had 1MB cache and its motherboard was also RCC based but a slightly older chipset. The Xeon had 256 bit wide memory compared to the 128 bit wide RCC HE chipset with PIII coppermine cpus. I think the Tyan board had two banks and could interleave for greater burst bandwidth. Anyway, for bang-per-buck, it was very impressive alongside a quad Xeon monster. Cheers, -Peter -- Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 3:41: 3 2000 Delivered-To: freebsd-smp@freebsd.org Received: from relay.butya.kz (butya-gw.butya.kz [212.154.129.94]) by hub.freebsd.org (Postfix) with ESMTP id 9830937B502; Thu, 5 Oct 2000 03:40:47 -0700 (PDT) Received: by relay.butya.kz (Postfix, from userid 1000) id 8C61728A0F; Thu, 5 Oct 2000 17:40:41 +0700 (ALMST) Received: from localhost (localhost [127.0.0.1]) by relay.butya.kz (Postfix) with ESMTP id 7A962287DF; Thu, 5 Oct 2000 17:40:41 +0700 (ALMST) Date: Thu, 5 Oct 2000 17:40:41 +0700 (ALMST) From: Boris Popov To: freebsd-current@freebsd.org Cc: freebsd-smp@freebsd.org Subject: Problems with kthread_exit() and SMPng Message-ID: MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="0-1753296439-970742441=:43806" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. Send mail to mime@docserver.cac.washington.edu for more info. --0-1753296439-970742441=:43806 Content-Type: TEXT/PLAIN; charset=US-ASCII Hello, Currently I'm trying to make KLD which uses kernel threads unloadable under recent -current. The prototype of functions looks like this: void my_thread(void*arg) { while(wearewanted) { do_something(); tsleep(); } exited = 1; kthread_exit(0); } void my_unload() { wearewanted = 0; while (!exited) tsleep(1sec); } my_unload() function called from the module unload event which issued from the kldunload() syscall. Unfortunately, kernel panics in the mtx_exit_hard() function. After some examination I've found that two fields in the Giant mutex structure set to unexpected values: ---- empty mtx_blocked for mutex Giant mtx_contested not in any list for mutex Giant ---- These messages printed by added diagnostics code. With this patch (see attachment) it is possible to load and unload KLD without any problems on UP machine except that the above messages printed. However, I'm don't know if they are correct. (btw, 4.1 doesn't have this problem). Any ideas why this happened and how to fix it ? -- Boris Popov http://www.butya.kz/~bp/ --0-1753296439-970742441=:43806 Content-Type: TEXT/PLAIN; charset=US-ASCII; name="synch_machdep.diff" Content-Transfer-Encoding: BASE64 Content-ID: Content-Description: Content-Disposition: attachment; filename="synch_machdep.diff" SW5kZXg6IHN5bmNoX21hY2hkZXAuYw0KPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PQ0KUkNTIGZpbGU6IC9ob21lL25jdnMvc3JjL3N5cy9pMzg2L2kzODYvc3lu Y2hfbWFjaGRlcC5jLHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS41DQpkaWZm IC11IC1yMS41IHN5bmNoX21hY2hkZXAuYw0KLS0tIHN5bmNoX21hY2hkZXAu YwkyMDAwLzEwLzA0IDAxOjIwOjQ5CTEuNQ0KKysrIHN5bmNoX21hY2hkZXAu YwkyMDAwLzEwLzA1IDEwOjIzOjMzDQpAQCAtMzU1LDEwICszNTUsMTYgQEAN CiAJCXAgPSBDVVJQUk9DOw0KIAkJcDEgPSBUQUlMUV9GSVJTVCgmbS0+bXR4 X2Jsb2NrZWQpOw0KIAkJTVBBU1MocC0+cF9tYWdpYyA9PSBQX01BR0lDKTsN Ci0JCU1QQVNTKHAxLT5wX21hZ2ljID09IFBfTUFHSUMpOw0KLQkJVEFJTFFf UkVNT1ZFKCZtLT5tdHhfYmxvY2tlZCwgcDEsIHBfcHJvY3EpOw0KKwkJaWYg KHAxKSB7DQorCQkJTVBBU1MocDEtPnBfbWFnaWMgPT0gUF9NQUdJQyk7DQor CQkJVEFJTFFfUkVNT1ZFKCZtLT5tdHhfYmxvY2tlZCwgcDEsIHBfcHJvY3Ep Ow0KKwkJfSBlbHNlDQorCQkJcHJpbnRmKCJlbXB0eSBtdHhfYmxvY2tlZCBm b3IgbXV0ZXggJXNcbiIsIG0tPm10eF9kZXNjcmlwdGlvbik7DQogCQlpZiAo VEFJTFFfRU1QVFkoJm0tPm10eF9ibG9ja2VkKSkgew0KLQkJCUxJU1RfUkVN T1ZFKG0sIG10eF9jb250ZXN0ZWQpOw0KKwkJCWlmIChtLT5tdHhfY29udGVz dGVkLmxlX3ByZXYgIT0gTlVMTCkNCisJCQkJTElTVF9SRU1PVkUobSwgbXR4 X2NvbnRlc3RlZCk7DQorCQkJZWxzZQ0KKwkJCQlwcmludGYoIm10eF9jb250 ZXN0ZWQgbm90IGluIGFueSBsaXN0IGZvciBtdXRleCAlc1xuIiwgbS0+bXR4 X2Rlc2NyaXB0aW9uKTsNCiAJCQlhdG9taWNfY21wc2V0X2ludCgmbS0+bXR4 X2xvY2ssIG0tPm10eF9sb2NrLA0KIAkJCQkJICBNVFhfVU5PV05FRCk7DQog CQkJQ1RSMShLVFJfTE9DSywgIm10eF9leGl0OiAweCVwIG5vdCBoZWxkIiwg bSk7DQpAQCAtMzczLDEyICszNzksMTUgQEANCiAJCWlmIChwcmkgPiBwLT5w X25hdGl2ZXByaSkNCiAJCQlwcmkgPSBwLT5wX25hdGl2ZXByaTsNCiAJCVNF VF9QUklPKHAsIHByaSk7DQotCQlDVFIyKEtUUl9MT0NLLCAibXR4X2V4aXQ6 IDB4JXAgY29udGVzdGVkIHNldHJ1bnF1ZXVlIDB4JXAiLA0KLQkJICAgIG0s IHAxKTsNCi0JCXAxLT5wX2Jsb2NrZWQgPSBOVUxMOw0KLQkJcDEtPnBfc3Rh dCA9IFNSVU47DQotCQlzZXRydW5xdWV1ZShwMSk7DQotCQlpZiAoKHR5cGUg JiBNVFhfTk9TV0lUQ0gpID09IDAgJiYgcDEtPnBfcHJpb3JpdHkgPCBwcmkp IHsNCisJCWlmIChwMSkgew0KKwkJCUNUUjIoS1RSX0xPQ0ssICJtdHhfZXhp dDogMHglcCBjb250ZXN0ZWQgc2V0cnVucXVldWUgMHglcCIsDQorCQkJICAg IG0sIHAxKTsNCisJCQlwMS0+cF9ibG9ja2VkID0gTlVMTDsNCisJCQlwMS0+ cF9zdGF0ID0gU1JVTjsNCisJCQlzZXRydW5xdWV1ZShwMSk7DQorCQl9DQor CQlpZiAoKHR5cGUgJiBNVFhfTk9TV0lUQ0gpID09IDAgJiYNCisJCSAgICAo cDEgPT0gTlVMTCB8fCBwMS0+cF9wcmlvcml0eSA8IHByaSkpIHsNCiAjaWZk ZWYgbm90eWV0DQogCQkJaWYgKHAtPnBfZmxhZyAmIChQX0lUSEQgfCBQX1NJ VEhEKSkgew0KIAkJCQlpdGhkX3QgKml0ID0gKGl0aGRfdCAqKXA7DQo= --0-1753296439-970742441=:43806-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 4:49:12 2000 Delivered-To: freebsd-smp@freebsd.org Received: from lists01.iafrica.com (lists01.iafrica.com [196.7.0.141]) by hub.freebsd.org (Postfix) with ESMTP id 8E32637B503 for ; Thu, 5 Oct 2000 04:49:08 -0700 (PDT) Received: from nwl.fw.uunet.co.za ([196.31.2.162]) by lists01.iafrica.com with esmtp (Exim 3.12 #2) id 13h9VT-00069w-00; Thu, 05 Oct 2000 13:48:11 +0200 Received: (from nobody@localhost) by nwl.fw.uunet.co.za (8.8.8/8.6.9) id NAA18006; Thu, 5 Oct 2000 13:48:14 +0200 (SAST) Received: by nwl.fw.uunet.co.za via recvmail id 17865; Thu Oct 5 13:47:35 2000 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.1/8.11.1) with ESMTP id e95BlZL04193; Thu, 5 Oct 2000 13:47:38 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200010051147.e95BlZL04193@grimreaper.grondar.za> To: Boris Popov Cc: freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng References: In-Reply-To: ; from Boris Popov "Thu, 05 Oct 2000 17:40:41 +0700." Date: Thu, 05 Oct 2000 13:47:34 +0200 From: Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Currently I'm trying to make KLD which uses kernel threads > unloadable under recent -current. I have exactly the same problem with the devrandom driver. I'll test your patch with my driver. Thanks! M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 8:55:17 2000 Delivered-To: freebsd-smp@freebsd.org Received: from io.yi.org (24.67.218.234.bc.wave.home.com [24.67.218.234]) by hub.freebsd.org (Postfix) with ESMTP id 194A037B503 for ; Thu, 5 Oct 2000 08:55:09 -0700 (PDT) Received: from io.yi.org (localhost.gvcl1.bc.wave.home.com [127.0.0.1]) by io.yi.org (Postfix) with ESMTP id 24CD9BA76; Thu, 5 Oct 2000 08:55:09 -0700 (PDT) X-Mailer: exmh version 2.1.1 10/15/1999 To: Mark Murray Cc: Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng In-Reply-To: Message from Mark Murray of "Thu, 05 Oct 2000 13:47:34 +0200." <200010051147.e95BlZL04193@grimreaper.grondar.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Oct 2000 08:55:09 -0700 From: Jake Burkholder Message-Id: <20001005155509.24CD9BA76@io.yi.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Currently I'm trying to make KLD which uses kernel threads > > unloadable under recent -current. > > I have exactly the same problem with the devrandom driver. I'll > test your patch with my driver. > > Thanks! > I'll bet that you just need to hold Giant before calling kthread_exit(), since exit1() isn't MP-safe yet. Running kernel threads outside of the lock is pretty iffy right now. void my_thread(void*arg) { while(wearewanted) { do_something(); tsleep(); } exited = 1; mtx_enter(&Giant, MTX_DEF); kthread_exit(0); } This should work, but depending on what you're doing it might be better to grab Giant as the first thing the thread does. Jake To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:10:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 861AF37B502 for ; Thu, 5 Oct 2000 11:10:49 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95IAZi08840; Thu, 5 Oct 2000 11:10:35 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010010331.VAA14982@harmony.village.org> Date: Thu, 05 Oct 2000 11:10:49 -0700 (PDT) From: John Baldwin To: Warner Losh Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 01-Oct-00 Warner Losh wrote: > In message John Baldwin writes: >: Grr. I'd test it on my laptop, but pccard isnt' attaching to my >: pcic_pci controller. I've tried the following patch but no go: > > The reason is that the pcic_p.c isn't putting the cardbus bridge into > legacy mode correctly. I'd bet a case of beer that the patch you > included won't work. Argh. I figured it out. I didn't have the pcic hints setup. I'd expect the pcic-pci driver to attach a pcic child just like the atapci driver attaches an ata child. :( Anyway we can get this fixed, or is it already on the todo list? >: pcic-pci0: at device 4.0 on pci0 > > These are always a pita. what kind of laptop do you have? > > Warner -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:15:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 032F337B502; Thu, 5 Oct 2000 11:15:41 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id e95IFcK00489; Thu, 5 Oct 2000 12:15:38 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id MAA00860; Thu, 5 Oct 2000 12:15:32 -0600 (MDT) Message-Id: <200010051815.MAA00860@harmony.village.org> To: John Baldwin Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org In-reply-to: Your message of "Thu, 05 Oct 2000 11:10:49 PDT." References: Date: Thu, 05 Oct 2000 12:15:32 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message John Baldwin writes: : On 01-Oct-00 Warner Losh wrote: : > In message John Baldwin writes: : >: Grr. I'd test it on my laptop, but pccard isnt' attaching to my : >: pcic_pci controller. I've tried the following patch but no go: : > : > The reason is that the pcic_p.c isn't putting the cardbus bridge into : > legacy mode correctly. I'd bet a case of beer that the patch you : > included won't work. : : Argh. I figured it out. I didn't have the pcic hints setup. I'd : expect the pcic-pci driver to attach a pcic child just like the : atapci driver attaches an ata child. :( Anyway we can get this fixed, : or is it already on the todo list? I don't care about bugs in OLDCARD that are at this level. This is a fundamental breakage in OLDCARD, imho. NEWCARD will eventually have a cardbus bridge driver that talks to the cardbus bridge chip using the yenta protocol (which defines a memory mapped version of the i82365SL register set, so there's opportunity for code sharing), rather than relying on the pci driver to put the device into a mode the isa driver can recognize. I also noticed that the interrupt process didn't go away when I ejected a pccard. I inserted another card and this process seemed to keep the new card from working. Ideas? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:27:10 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id 4BAD937B66C for ; Thu, 5 Oct 2000 11:27:04 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13hFjF-0008UJ-00; Thu, 5 Oct 2000 20:26:50 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.1/8.11.1) with ESMTP id e95IQfJ12260; Thu, 5 Oct 2000 20:26:53 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200010051826.e95IQfJ12260@grimreaper.grondar.za> To: Jake Burkholder Cc: Mark Murray , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng References: <20001005155509.24CD9BA76@io.yi.org> In-Reply-To: <20001005155509.24CD9BA76@io.yi.org> ; from Jake Burkholder "Thu, 05 Oct 2000 08:55:09 MST." Date: Thu, 05 Oct 2000 20:26:41 +0200 From: Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I'll bet that you just need to hold Giant before calling > kthread_exit(), since exit1() isn't MP-safe yet. Running > kernel threads outside of the lock is pretty iffy right now. > > void > my_thread(void*arg) > { > while(wearewanted) { > do_something(); > tsleep(); > } > exited = 1; > mtx_enter(&Giant, MTX_DEF); > kthread_exit(0); > } > > This should work, but depending on what you're doing it might > be better to grab Giant as the first thing the thread does. Hmm. I'll try that. Will the (kthread_)exit release Giant? My thread does a lot of work; if it holds Giant, then it can't be preempted. That would be a problem. There is another problem; printf's inside a kthread corrupt like crazy. They look very unthreadsafe. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:32: 4 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id D818A37B66D for ; Thu, 5 Oct 2000 11:32:01 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e95IVda01048; Thu, 5 Oct 2000 11:31:39 -0700 (PDT) Date: Thu, 5 Oct 2000 11:31:39 -0700 From: Alfred Perlstein To: Mark Murray Cc: Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng Message-ID: <20001005113139.C27736@fw.wintelcom.net> References: <20001005155509.24CD9BA76@io.yi.org> <200010051826.e95IQfJ12260@grimreaper.grondar.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200010051826.e95IQfJ12260@grimreaper.grondar.za>; from mark@grondar.za on Thu, Oct 05, 2000 at 08:26:41PM +0200 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Mark Murray [001005 11:29] wrote: > > There is another problem; printf's inside a kthread corrupt like > crazy. They look very unthreadsafe. do NOT use printf without Giant. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:53:31 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.iafrica.com (smtp02.iafrica.com [196.7.0.140]) by hub.freebsd.org (Postfix) with ESMTP id C9D5637B503 for ; Thu, 5 Oct 2000 11:53:23 -0700 (PDT) Received: from [196.7.18.138] (helo=grimreaper.grondar.za ident=root) by smtp02.iafrica.com with esmtp (Exim 1.92 #1) id 13hG8t-0009Bz-00; Thu, 5 Oct 2000 20:53:19 +0200 Received: from grimreaper.grondar.za (mark@localhost [127.0.0.1]) by grimreaper.grondar.za (8.11.1/8.11.1) with ESMTP id e95IrPJ12665; Thu, 5 Oct 2000 20:53:25 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200010051853.e95IrPJ12665@grimreaper.grondar.za> To: Alfred Perlstein Cc: freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng References: <20001005113139.C27736@fw.wintelcom.net> In-Reply-To: <20001005113139.C27736@fw.wintelcom.net> ; from Alfred Perlstein "Thu, 05 Oct 2000 11:31:39 MST." Date: Thu, 05 Oct 2000 20:53:25 +0200 From: Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > * Mark Murray [001005 11:29] wrote: > > > > There is another problem; printf's inside a kthread corrupt like > > crazy. They look very unthreadsafe. > > do NOT use printf without Giant. ACK. Thanks! M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:54: 3 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 0122137B674 for ; Thu, 5 Oct 2000 11:53:53 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95Irbi09931; Thu, 5 Oct 2000 11:53:37 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010051815.MAA00860@harmony.village.org> Date: Thu, 05 Oct 2000 11:53:52 -0700 (PDT) From: John Baldwin To: Warner Losh Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 05-Oct-00 Warner Losh wrote: > In message John Baldwin writes: >: On 01-Oct-00 Warner Losh wrote: >: > In message John Baldwin writes: >: >: Grr. I'd test it on my laptop, but pccard isnt' attaching to my >: >: pcic_pci controller. I've tried the following patch but no go: >: > >: > The reason is that the pcic_p.c isn't putting the cardbus bridge into >: > legacy mode correctly. I'd bet a case of beer that the patch you >: > included won't work. >: >: Argh. I figured it out. I didn't have the pcic hints setup. I'd >: expect the pcic-pci driver to attach a pcic child just like the >: atapci driver attaches an ata child. :( Anyway we can get this fixed, >: or is it already on the todo list? > > I don't care about bugs in OLDCARD that are at this level. This is a > fundamental breakage in OLDCARD, imho. NEWCARD will eventually have a > cardbus bridge driver that talks to the cardbus bridge chip using the > yenta protocol (which defines a memory mapped version of the i82365SL > register set, so there's opportunity for code sharing), rather than > relying on the pci driver to put the device into a mode the isa driver > can recognize. > > I also noticed that the interrupt process didn't go away when I > ejected a pccard. I inserted another card and this process seemed to > keep the new card from working. Ideas? We don't have a kthread_cancel(), so there is no good way to kill the thread. However, the code should handle adding and removing handlers ok. In inthand_add() we check to see if the thread we are adding a handler to has no handlers and if not we act as if we are sort of creating a new ithread. > Warner -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:54:35 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 1768C37B502 for ; Thu, 5 Oct 2000 11:54:33 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95Irdi09935; Thu, 5 Oct 2000 11:53:39 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010051826.e95IQfJ12260@grimreaper.grondar.za> Date: Thu, 05 Oct 2000 11:53:53 -0700 (PDT) From: John Baldwin To: Mark Murray Subject: Re: Problems with kthread_exit() and SMPng Cc: freebsd-smp@FreeBSD.org, Boris Popov , Jake Burkholder Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 05-Oct-00 Mark Murray wrote: >> I'll bet that you just need to hold Giant before calling >> kthread_exit(), since exit1() isn't MP-safe yet. Running >> kernel threads outside of the lock is pretty iffy right now. >> >> void >> my_thread(void*arg) >> { >> while(wearewanted) { >> do_something(); >> tsleep(); >> } >> exited = 1; >> mtx_enter(&Giant, MTX_DEF); >> kthread_exit(0); >> } >> >> This should work, but depending on what you're doing it might >> be better to grab Giant as the first thing the thread does. > > Hmm. I'll try that. Will the (kthread_)exit release Giant? Yes. cpu_exit() (called by exit1()) releases Giant as its final act before calling cpu_switch() (should be cpu_throw()) to run the next process. > My thread does a lot of work; if it holds Giant, then it can't be > preempted. That would be a problem. Errr, it can be pre-empted, but your thread doesn't need Giant except for kthread_exit(). > There is another problem; printf's inside a kthread corrupt like > crazy. They look very unthreadsafe. printf() is not MP safe. :-P You probably want to grab Giant before calling printf(). > M > -- > Mark Murray > Join the anti-SPAM movement: http://www.cauce.org -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 11:56:19 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 95A8A37B66C; Thu, 5 Oct 2000 11:56:15 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id e95IuDK00675; Thu, 5 Oct 2000 12:56:14 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id MAA01219; Thu, 5 Oct 2000 12:56:13 -0600 (MDT) Message-Id: <200010051856.MAA01219@harmony.village.org> To: John Baldwin Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org In-reply-to: Your message of "Thu, 05 Oct 2000 11:53:52 PDT." References: Date: Thu, 05 Oct 2000 12:56:13 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message John Baldwin writes: : thread. However, the code should handle adding and removing handlers : ok. In inthand_add() we check to see if the thread we are adding a : handler to has no handlers and if not we act as if we are sort of creating : a new ithread. OK. I'll look at why inthand_add() isn't working in that case then, or track down why the interrupts don't get passed on to my interrupt routine. thanks for the pointer. I've seen it twice now, but usually don't see it because I usually just have one card. I'll need a kthread_cancel for unloading the pccard device in NEWCARD since it uses a thread to process the insert/remove events. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 12:14:33 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 8742837B503 for ; Thu, 5 Oct 2000 12:14:30 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95JEGi10737; Thu, 5 Oct 2000 12:14:17 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200010051856.MAA01219@harmony.village.org> Date: Thu, 05 Oct 2000 12:14:31 -0700 (PDT) From: John Baldwin To: Warner Losh Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 05-Oct-00 Warner Losh wrote: > In message John Baldwin writes: >: thread. However, the code should handle adding and removing handlers >: ok. In inthand_add() we check to see if the thread we are adding a >: handler to has no handlers and if not we act as if we are sort of creating >: a new ithread. > > OK. I'll look at why inthand_add() isn't working in that case then, > or track down why the interrupts don't get passed on to my interrupt > routine. thanks for the pointer. I've seen it twice now, but usually > don't see it because I usually just have one card. > > I'll need a kthread_cancel for unloading the pccard device in NEWCARD > since it uses a thread to process the insert/remove events. You may be able to orchestrate a way to have your thread call kthread_exit(), which would work. The person to bug for a kthread_cancel() is Peter though. :) > Warner -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 12:18:47 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 4B1D837B502; Thu, 5 Oct 2000 12:18:44 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id e95JIgK00736; Thu, 5 Oct 2000 13:18:43 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id NAA01346; Thu, 5 Oct 2000 13:18:42 -0600 (MDT) Message-Id: <200010051918.NAA01346@harmony.village.org> To: John Baldwin Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org In-reply-to: Your message of "Thu, 05 Oct 2000 12:14:31 PDT." References: Date: Thu, 05 Oct 2000 13:18:42 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message John Baldwin writes: : You may be able to orchestrate a way to have your thread call kthread_exit(), : which would work. The person to bug for a kthread_cancel() is Peter though. : :) OK. kthread_exit() is likely sufficient for my needs in NEWCARD. I can queue a poison pill as easily as I can queue an INSERT or REMOVED event :-) Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 13:11: 6 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp1-ext.oskarmobil.cz (smtp1-ext.oskarmobil.cz [195.47.29.137]) by hub.freebsd.org (Postfix) with ESMTP id 5C8FD37B503; Thu, 5 Oct 2000 13:11:03 -0700 (PDT) Received: from wh01ex02.ceskymobil.cz (exchange1.ceskymobil.cz [172.20.128.42]) by smtp1-ext.oskarmobil.cz (8.9.3/8.9.3) with ESMTP id WAA38207; Thu, 5 Oct 2000 22:07:49 +0200 (CEST) Received: by exchange1.ceskymobil.cz with Internet Mail Service (5.5.2650.21) id <4K1QW24Q>; Thu, 5 Oct 2000 22:10:46 +0200 Message-ID: <11CCE9D433CFD311B32A00508B95BCA00170C0BF@exchange1.ceskymobil.cz> From: =?iso-8859-1?Q?Milon_Papez=EDk?= To: "'Warner Losh'" , John Baldwin Cc: smp@FreeBSD.ORG Subject: RE: Releasing interrupts? Date: Thu, 5 Oct 2000 22:10:45 +0200 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2650.21) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi Warner, do I have to use current to get the NEWCARD changes or will I find them already MFC-ed in STABLE as well ? Thanks in advance, Milon -- milon.papezik@oskarmobil.cz -----Original Message----- From: Warner Losh [mailto:imp@village.org] Sent: Thursday, October 05, 2000 8:56 PM To: John Baldwin Cc: smp@FreeBSD.ORG Subject: Re: Releasing interrupts? In message John Baldwin writes: : thread. However, the code should handle adding and removing handlers : ok. In inthand_add() we check to see if the thread we are adding a : handler to has no handlers and if not we act as if we are sort of creating : a new ithread. OK. I'll look at why inthand_add() isn't working in that case then, or track down why the interrupts don't get passed on to my interrupt routine. thanks for the pointer. I've seen it twice now, but usually don't see it because I usually just have one card. I'll need a kthread_cancel for unloading the pccard device in NEWCARD since it uses a thread to process the insert/remove events. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 13:16: 2 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 6273C37B502; Thu, 5 Oct 2000 13:15:59 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id e95KFvK00904; Thu, 5 Oct 2000 14:15:58 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id OAA01625; Thu, 5 Oct 2000 14:15:57 -0600 (MDT) Message-Id: <200010052015.OAA01625@harmony.village.org> To: =?iso-8859-1?Q?Milon_Papez=EDk?= Subject: Re: Releasing interrupts? Cc: John Baldwin , smp@FreeBSD.ORG In-reply-to: Your message of "Thu, 05 Oct 2000 22:10:45 +0200." <11CCE9D433CFD311B32A00508B95BCA00170C0BF@exchange1.ceskymobil.cz> References: <11CCE9D433CFD311B32A00508B95BCA00170C0BF@exchange1.ceskymobil.cz> Date: Thu, 05 Oct 2000 14:15:57 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <11CCE9D433CFD311B32A00508B95BCA00170C0BF@exchange1.ceskymobil.cz> =?iso-8859-1?Q?Milon_Papez=EDk?= writes: : do I have to use current to get the NEWCARD changes : or will I find them already MFC-ed in STABLE as well ? You should use the OLDCARD code right now. The NEWCARD code is only for developers right now. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 13:37:22 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.du.gtn.com (mail.du.gtn.com [194.77.9.57]) by hub.freebsd.org (Postfix) with ESMTP id 6A7F637B66D; Thu, 5 Oct 2000 13:37:18 -0700 (PDT) Received: from mail.cicely.de (cicely.de [194.231.9.142]) by mail.du.gtn.com (8.11.0.Beta3/8.11.0.Beta3) with ESMTP id e95KbDC19120 (using TLSv1/SSLv3 with cipher EDH-RSA-DES-CBC3-SHA (168 bits) verified OK); Thu, 5 Oct 2000 22:37:16 +0200 (MET DST) Received: from cicely5.cicely.de (cicely5.cicely.de [fec0::104:200:92ff:fe9b:20e7]) by mail.cicely.de (8.11.0.Beta1/8.11.0.Beta1) with ESMTP id e95KbjT63205; Thu, 5 Oct 2000 22:37:45 +0200 (CEST) Received: (from ticso@localhost) by cicely5.cicely.de (8.11.0/8.9.2) id e95KbYC40750; Thu, 5 Oct 2000 22:37:34 +0200 (CEST) (envelope-from ticso) Date: Thu, 5 Oct 2000 22:37:33 +0200 From: Bernd Walter To: John Baldwin Cc: Mark Murray , freebsd-smp@FreeBSD.ORG, Boris Popov , Jake Burkholder Subject: Re: Problems with kthread_exit() and SMPng Message-ID: <20001005223733.A40689@cicely5.cicely.de> References: <200010051826.e95IQfJ12260@grimreaper.grondar.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: ; from jhb@FreeBSD.ORG on Thu, Oct 05, 2000 at 11:53:53AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, Oct 05, 2000 at 11:53:53AM -0700, John Baldwin wrote: > > On 05-Oct-00 Mark Murray wrote: > > Hmm. I'll try that. Will the (kthread_)exit release Giant? > > Yes. cpu_exit() (called by exit1()) releases Giant as its > final act before calling cpu_switch() (should be cpu_throw()) > to run the next process. > > > My thread does a lot of work; if it holds Giant, then it can't be > > preempted. That would be a problem. > > Errr, it can be pre-empted, but your thread doesn't need Giant except > for kthread_exit(). > > > There is another problem; printf's inside a kthread corrupt like > > crazy. They look very unthreadsafe. > > printf() is not MP safe. :-P You probably want to grab Giant before > calling printf(). Isn't it better to just fetch Giant inside of kthread_exit() and printf()? Giant is recursive so it shouldn't hurt much if we allocate it while already owning it but we don't need to rechange thread safe code after these restrictions fall. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 14:31:10 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 62CBA37B502 for ; Thu, 5 Oct 2000 14:31:07 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95LURi15463; Thu, 5 Oct 2000 14:30:27 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20001005223733.A40689@cicely5.cicely.de> Date: Thu, 05 Oct 2000 14:30:39 -0700 (PDT) From: John Baldwin To: Bernd Walter Subject: Re: Problems with kthread_exit() and SMPng Cc: Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.org, Mark Murray Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 05-Oct-00 Bernd Walter wrote: > On Thu, Oct 05, 2000 at 11:53:53AM -0700, John Baldwin wrote: >> >> On 05-Oct-00 Mark Murray wrote: >> > Hmm. I'll try that. Will the (kthread_)exit release Giant? >> >> Yes. cpu_exit() (called by exit1()) releases Giant as its >> final act before calling cpu_switch() (should be cpu_throw()) >> to run the next process. >> >> > My thread does a lot of work; if it holds Giant, then it can't be >> > preempted. That would be a problem. >> >> Errr, it can be pre-empted, but your thread doesn't need Giant except >> for kthread_exit(). >> >> > There is another problem; printf's inside a kthread corrupt like >> > crazy. They look very unthreadsafe. >> >> printf() is not MP safe. :-P You probably want to grab Giant before >> calling printf(). > > Isn't it better to just fetch Giant inside of kthread_exit() and printf()? > Giant is recursive so it shouldn't hurt much if we allocate it while > already owning it but we don't need to rechange thread safe code after > these restrictions fall. Ugh. If I'm going to go stick mutexes in code, I want to put the right ones in. :-P > -- > B.Walter COSMO-Project http://www.cosmo-project.de > ticso@cicely.de Usergroup info@cosmo-project.de -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 14:31:16 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 2A72537B66E; Thu, 5 Oct 2000 14:31:10 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95LUsi15499; Thu, 5 Oct 2000 14:30:54 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Thu, 05 Oct 2000 14:31:07 -0700 (PDT) From: John Baldwin To: Boris Popov Subject: RE: Problems with kthread_exit() and SMPng Cc: freebsd-smp@FreeBSD.org, freebsd-current@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 05-Oct-00 Boris Popov wrote: > Hello, > > Currently I'm trying to make KLD which uses kernel threads > unloadable under recent -current. The prototype of functions looks like > this: > > void > my_thread(void*arg) > { > while(wearewanted) { > do_something(); > tsleep(); > } > exited = 1; > kthread_exit(0); > } > > void > my_unload() > { > wearewanted = 0; > while (!exited) > tsleep(1sec); > } > > my_unload() function called from the module unload event which > issued from the kldunload() syscall. > > Unfortunately, kernel panics in the mtx_exit_hard() function. > After some examination I've found that two fields in the Giant mutex > structure set to unexpected values: > ---- > empty mtx_blocked for mutex Giant > mtx_contested not in any list for mutex Giant > ---- > > These messages printed by added diagnostics code. With this patch > (see attachment) it is possible to load and unload KLD without any > problems on UP machine except that the above messages printed. However, > I'm don't know if they are correct. (btw, 4.1 doesn't have this problem). > > Any ideas why this happened and how to fix it ? > -- > Boris Popov > http://www.butya.kz/~bp/ -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 14:31:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id E2E0737B671; Thu, 5 Oct 2000 14:31:10 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95LUti15503; Thu, 5 Oct 2000 14:30:55 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Thu, 05 Oct 2000 14:31:07 -0700 (PDT) From: John Baldwin To: Boris Popov Subject: RE: Problems with kthread_exit() and SMPng Cc: freebsd-smp@FreeBSD.org, freebsd-current@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 05-Oct-00 Boris Popov wrote: > Hello, > > Currently I'm trying to make KLD which uses kernel threads > unloadable under recent -current. The prototype of functions looks like > this: > > void > my_thread(void*arg) > { > while(wearewanted) { > do_something(); > tsleep(); > } > exited = 1; > kthread_exit(0); > } You need to have Giant before you call kthread_exit(). > my_unload() function called from the module unload event which > issued from the kldunload() syscall. > > Unfortunately, kernel panics in the mtx_exit_hard() function. > After some examination I've found that two fields in the Giant mutex > structure set to unexpected values: Ah, I need to add a KASSERT() to the start of mtx_exit_hard() to verify that we own the mutex we are releasing. > ---- > empty mtx_blocked for mutex Giant > mtx_contested not in any list for mutex Giant > ---- > > These messages printed by added diagnostics code. With this patch > (see attachment) it is possible to load and unload KLD without any > problems on UP machine except that the above messages printed. However, > I'm don't know if they are correct. (btw, 4.1 doesn't have this problem). It is a bogus patch though. If you release a contested mutex, it should always have someone who is waiting to grab it. > Any ideas why this happened and how to fix it ? > -- > Boris Popov > http://www.butya.kz/~bp/ -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 14:42:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id C2DEC37B502 for ; Thu, 5 Oct 2000 14:42:46 -0700 (PDT) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id OAA26992; Thu, 5 Oct 2000 14:43:03 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp05.primenet.com, id smtpdAAAdbaWL0; Thu Oct 5 14:42:55 2000 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id OAA15421; Thu, 5 Oct 2000 14:42:29 -0700 (MST) From: Terry Lambert Message-Id: <200010052142.OAA15421@usr05.primenet.com> Subject: Re: Problems with kthread_exit() and SMPng To: bright@wintelcom.net (Alfred Perlstein) Date: Thu, 5 Oct 2000 21:42:28 +0000 (GMT) Cc: mark@grondar.za (Mark Murray), jburkhol@home.com (Jake Burkholder), bp@butya.kz (Boris Popov), freebsd-smp@FreeBSD.ORG In-Reply-To: <20001005113139.C27736@fw.wintelcom.net> from "Alfred Perlstein" at Oct 05, 2000 11:31:39 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > There is another problem; printf's inside a kthread corrupt like > > crazy. They look very unthreadsafe. > > do NOT use printf without Giant. This strikes me as being rather inane. If printf won't work without holging the lock, then it damn well should acquire the lock if it isn't already held, and release it if it acquired it, before returning. This is not something that a programmer should have to worry about when they use a service; unless you are doing threads IPC, perhaps in some vainglorious attempt to make SMP systems stall their little butts off with interprocessor synchrnization, this should never be an issue. In other words, I don't give a damn, as a programmer, whether a service is reentrant or not, so long as it works when I call it. Having the lock fairy sprinkle lock primitives before and after every service call is not my idea of a Good Thing. This appears to be a very good point to note that having an owner or recursing on this lock would both end up being Very Bad Things, since it would make the obvious and intuitive (and therefore least astonishing) behaviour impossible. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 14:46:56 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 5BFD737B503; Thu, 5 Oct 2000 14:46:47 -0700 (PDT) Received: from laptop.baldwin.cx (john@dhcp248.osd.bsdi.com [204.216.28.248]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e95Lkai16181; Thu, 5 Oct 2000 14:46:36 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Thu, 05 Oct 2000 14:46:49 -0700 (PDT) From: John Baldwin To: John Baldwin Subject: RE: Problems with kthread_exit() and SMPng Cc: freebsd-current@FreeBSD.org, freebsd-smp@FreeBSD.org, Boris Popov Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Ugh, my mail client ate my reply, lemme try again. > On 05-Oct-00 Boris Popov wrote: >> Hello, >> >> Currently I'm trying to make KLD which uses kernel threads >> unloadable under recent -current. The prototype of functions looks like >> this: >> >> void >> my_thread(void*arg) >> { >> while(wearewanted) { >> do_something(); >> tsleep(); >> } >> exited = 1; >> kthread_exit(0); >> } You need Giant before calling kthread_exit(). >> void >> my_unload() >> { >> wearewanted = 0; >> while (!exited) >> tsleep(1sec); >> } >> >> my_unload() function called from the module unload event which >> issued from the kldunload() syscall. >> >> Unfortunately, kernel panics in the mtx_exit_hard() function. >> After some examination I've found that two fields in the Giant mutex >> structure set to unexpected values: It should have died much earlier if you had INVARIANTS turned on. :( It looks like you are releasing a mutex you probably do not own because cpu_exit() (called by exit1() -> exit() -> kthread_exit()) releases Giant as one of its final tasks. >> ---- >> empty mtx_blocked for mutex Giant >> mtx_contested not in any list for mutex Giant >> ---- >> >> These messages printed by added diagnostics code. With this patch >> (see attachment) it is possible to load and unload KLD without any >> problems on UP machine except that the above messages printed. However, >> I'm don't know if they are correct. (btw, 4.1 doesn't have this problem). This patch is bogus I'm afraid. A contested mutex should always have a process waiting to grab it when it is released. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 15:15:29 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 4F7BB37B502 for ; Thu, 5 Oct 2000 15:15:26 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e95MF5U08444; Thu, 5 Oct 2000 15:15:05 -0700 (PDT) Date: Thu, 5 Oct 2000 15:15:04 -0700 From: Alfred Perlstein To: Terry Lambert Cc: Mark Murray , Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng Message-ID: <20001005151504.K27736@fw.wintelcom.net> References: <20001005113139.C27736@fw.wintelcom.net> <200010052142.OAA15421@usr05.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200010052142.OAA15421@usr05.primenet.com>; from tlambert@primenet.com on Thu, Oct 05, 2000 at 09:42:28PM +0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Terry Lambert [001005 14:42] wrote: > > > There is another problem; printf's inside a kthread corrupt like > > > crazy. They look very unthreadsafe. > > > > do NOT use printf without Giant. > > This strikes me as being rather inane. > > If printf won't work without holging the lock, then it damn well > should acquire the lock if it isn't already held, and release it > if it acquired it, before returning. > > This is not something that a programmer should have to worry > about when they use a service; unless you are doing threads > IPC, perhaps in some vainglorious attempt to make SMP systems > stall their little butts off with interprocessor synchrnization, > this should never be an issue. > > In other words, I don't give a damn, as a programmer, whether > a service is reentrant or not, so long as it works when I call > it. Having the lock fairy sprinkle lock primitives before and > after every service call is not my idea of a Good Thing. > > > This appears to be a very good point to note that having an > owner or recursing on this lock would both end up being Very > Bad Things, since it would make the obvious and intuitive > (and therefore least astonishing) behaviour impossible. One of the options for fixing kernel logging under SMP was to have a very simple queing mechanism to send the messages to a kernel thread that would be repsoncible for the output. You see, the problem with logging is that the logging calls are in a lot of weird places, nevermind critical ones like panic, there needs to be a special relationship between the logging kthread and the system which may be difficult to set up. You also have to take into account that the output method of the driver might block therefore we need to hand off the blocking to someone else. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 15:21:45 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.du.gtn.com (mail.du.gtn.com [194.77.9.57]) by hub.freebsd.org (Postfix) with ESMTP id 0EEF037B503; Thu, 5 Oct 2000 15:21:38 -0700 (PDT) Received: from mail.cicely.de (cicely.de [194.231.9.142]) by mail.du.gtn.com (8.11.0.Beta3/8.11.0.Beta3) with ESMTP id e95MLYq28519 (using TLSv1/SSLv3 with cipher EDH-RSA-DES-CBC3-SHA (168 bits) verified OK); Fri, 6 Oct 2000 00:21:36 +0200 (MET DST) Received: from cicely5.cicely.de (cicely5.cicely.de [fec0::104:200:92ff:fe9b:20e7]) by mail.cicely.de (8.11.0.Beta1/8.11.0.Beta1) with ESMTP id e95MM6T63554; Fri, 6 Oct 2000 00:22:06 +0200 (CEST) Received: (from ticso@localhost) by cicely5.cicely.de (8.11.0/8.9.2) id e95MM1P41392; Fri, 6 Oct 2000 00:22:01 +0200 (CEST) (envelope-from ticso) Date: Fri, 6 Oct 2000 00:22:01 +0200 From: Bernd Walter To: John Baldwin Cc: Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.org, Mark Murray Subject: Re: Problems with kthread_exit() and SMPng Message-ID: <20001006002200.E40689@cicely5.cicely.de> References: <20001005223733.A40689@cicely5.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: ; from jhb@FreeBSD.org on Thu, Oct 05, 2000 at 02:30:39PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, Oct 05, 2000 at 02:30:39PM -0700, John Baldwin wrote: > > On 05-Oct-00 Bernd Walter wrote: > > On Thu, Oct 05, 2000 at 11:53:53AM -0700, John Baldwin wrote: > >> > >> On 05-Oct-00 Mark Murray wrote: > >> > Hmm. I'll try that. Will the (kthread_)exit release Giant? > >> > >> Yes. cpu_exit() (called by exit1()) releases Giant as its > >> final act before calling cpu_switch() (should be cpu_throw()) > >> to run the next process. > >> > >> > My thread does a lot of work; if it holds Giant, then it can't be > >> > preempted. That would be a problem. > >> > >> Errr, it can be pre-empted, but your thread doesn't need Giant except > >> for kthread_exit(). > >> > >> > There is another problem; printf's inside a kthread corrupt like > >> > crazy. They look very unthreadsafe. > >> > >> printf() is not MP safe. :-P You probably want to grab Giant before > >> calling printf(). > > > > Isn't it better to just fetch Giant inside of kthread_exit() and printf()? > > Giant is recursive so it shouldn't hurt much if we allocate it while > > already owning it but we don't need to rechange thread safe code after > > these restrictions fall. > > Ugh. If I'm going to go stick mutexes in code, I want to put the > right ones in. :-P Of course that's much better ;) The situation with such functions is not uncritical as you may need to release other mutexes before you call printf(); Say you have a driver which is called via another driver (upcall, ...). The other driver is not updated and calls us while holding Giant and we need to fetch one of our own mutexes for processing. That means we have to own Giant before we fetch our own mutex if we want to make use of printf() or we need to release our before entering Giant to do printf(). Otherwise we don't have a fixed mutex order :( -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 18:34:55 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 1279037B502 for ; Thu, 5 Oct 2000 18:34:52 -0700 (PDT) Received: (qmail 98454 invoked by uid 1142); 6 Oct 2000 01:34:51 -0000 Date: 5 Oct 2000 18:34:51 -0700 Date: Thu, 5 Oct 2000 16:08:13 -0700 From: Jason Evans To: Warner Losh Cc: John Baldwin , smp@FreeBSD.org Subject: Re: Releasing interrupts? Message-ID: <20001005160813.R58256@canonware.com> References: <200010051918.NAA01346@harmony.village.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200010051918.NAA01346@harmony.village.org>; from imp@village.org on Thu, Oct 05, 2000 at 01:18:42PM -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, Oct 05, 2000 at 01:18:42PM -0600, Warner Losh wrote: > In message John Baldwin writes: > : You may be able to orchestrate a way to have your thread call kthread_exit(), > : which would work. The person to bug for a kthread_cancel() is Peter though. > : :) > > OK. kthread_exit() is likely sufficient for my needs in NEWCARD. I > can queue a poison pill as easily as I can queue an INSERT or REMOVED > event :-) Good to hear it. With regard to thread cancellation, it's another example of a feature that, if used, usually indicates poor program design, so I hope that we don't find a need for kthread_cancel(). Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 18:39: 0 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 96D6A37B503; Thu, 5 Oct 2000 18:38:57 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.11.0/8.11.0) with ESMTP id e961csK02058; Thu, 5 Oct 2000 19:38:55 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id TAA03855; Thu, 5 Oct 2000 19:38:54 -0600 (MDT) Message-Id: <200010060138.TAA03855@harmony.village.org> To: Jason Evans Subject: Re: Releasing interrupts? Cc: John Baldwin , smp@FreeBSD.org In-reply-to: Your message of "Thu, 05 Oct 2000 16:08:13 PDT." <20001005160813.R58256@canonware.com> References: <20001005160813.R58256@canonware.com> <200010051918.NAA01346@harmony.village.org> Date: Thu, 05 Oct 2000 19:38:54 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <20001005160813.R58256@canonware.com> Jason Evans writes: : Good to hear it. With regard to thread cancellation, it's another example : of a feature that, if used, usually indicates poor program design, so I : hope that we don't find a need for kthread_cancel(). Even for the interrupt threads? I'd think you'd want to kill them when their reference count goes to zero. It certainly looks weird in the ps listing to see ed1 on irq 11 when you know you ejected the card... Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 18:53:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sasami.jurai.net (sasami.jurai.net [63.67.141.99]) by hub.freebsd.org (Postfix) with ESMTP id 78FAB37B502 for ; Thu, 5 Oct 2000 18:53:46 -0700 (PDT) Received: from localhost (winter@localhost) by sasami.jurai.net (8.9.3/8.8.7) with ESMTP id VAA02438; Thu, 5 Oct 2000 21:53:11 -0400 (EDT) Date: Thu, 5 Oct 2000 21:53:11 -0400 (EDT) From: "Matthew N. Dodd" To: Mark Murray Cc: Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng In-Reply-To: <200010051826.e95IQfJ12260@grimreaper.grondar.za> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 5 Oct 2000, Mark Murray wrote: > There is another problem; printf's inside a kthread corrupt like > crazy. They look very unthreadsafe. I've been thinking about a buffered printf. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | This Space For Rent | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 18:55:41 2000 Delivered-To: freebsd-smp@freebsd.org Received: from relay.butya.kz (butya-gw.butya.kz [212.154.129.94]) by hub.freebsd.org (Postfix) with ESMTP id D8E4D37B503; Thu, 5 Oct 2000 18:55:35 -0700 (PDT) Received: by relay.butya.kz (Postfix, from userid 1000) id B93CB28AB3; Fri, 6 Oct 2000 08:55:14 +0700 (ALMST) Received: from localhost (localhost [127.0.0.1]) by relay.butya.kz (Postfix) with ESMTP id A6B8828AAF; Fri, 6 Oct 2000 08:55:14 +0700 (ALMST) Date: Fri, 6 Oct 2000 08:55:14 +0700 (ALMST) From: Boris Popov To: John Baldwin Cc: freebsd-current@FreeBSD.org, freebsd-smp@FreeBSD.org Subject: RE: Problems with kthread_exit() and SMPng In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 5 Oct 2000, John Baldwin wrote: > You need Giant before calling kthread_exit(). Ok. > >> After some examination I've found that two fields in the Giant mutex > >> structure set to unexpected values: > > It should have died much earlier if you had INVARIANTS turned on. :( It > looks like you are releasing a mutex you probably do not own because > cpu_exit() (called by exit1() -> exit() -> kthread_exit()) releases Giant > as one of its final tasks. This is probably the bug somewhere in the diagnostic code. I have INVARIANTS/INVARIANT_SUPPORT/DIAGNOSTIC turned on and UP machine just panics in the mtx_exit_hard() while SMP machine silently reboots :( > This patch is bogus I'm afraid. A contested mutex should always have a > process waiting to grab it when it is released. Yes, it should be bogus. But diagnostic is rather useful :) -- Boris Popov http://www.butya.kz/~bp/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 18:56:55 2000 Delivered-To: freebsd-smp@freebsd.org Received: from relay.butya.kz (butya-gw.butya.kz [212.154.129.94]) by hub.freebsd.org (Postfix) with ESMTP id C4CF737B502 for ; Thu, 5 Oct 2000 18:56:52 -0700 (PDT) Received: by relay.butya.kz (Postfix, from userid 1000) id 6311728AAF; Fri, 6 Oct 2000 08:56:50 +0700 (ALMST) Received: from localhost (localhost [127.0.0.1]) by relay.butya.kz (Postfix) with ESMTP id 5884D28645; Fri, 6 Oct 2000 08:56:50 +0700 (ALMST) Date: Fri, 6 Oct 2000 08:56:50 +0700 (ALMST) From: Boris Popov To: Jake Burkholder Cc: Mark Murray , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng In-Reply-To: <20001005155509.24CD9BA76@io.yi.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 5 Oct 2000, Jake Burkholder wrote: > I'll bet that you just need to hold Giant before calling > kthread_exit(), since exit1() isn't MP-safe yet. Running > kernel threads outside of the lock is pretty iffy right now. Thanks, this helps (however adds more #ifdefs for code portable between releng_4 and -current). -- Boris Popov http://www.butya.kz/~bp/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 19:45: 4 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 5A5F437B502 for ; Thu, 5 Oct 2000 19:45:02 -0700 (PDT) Received: from berserker.bsdi.com (cp@localhost [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id UAA09999; Thu, 5 Oct 2000 20:44:53 -0600 (MDT) Message-Id: <200010060244.UAA09999@berserker.bsdi.com> To: "Matthew N. Dodd" Cc: freebsd-smp@freebsd.org Subject: Re: Problems with kthread_exit() and SMPng In-reply-to: Your message of "Thu, 05 Oct 2000 21:53:11 EDT." From: Chuck Paterson Date: Thu, 05 Oct 2000 20:44:53 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Matthew, I actaully have started down this path, but other work keeps getting in the way. I'm actually feeling kind of guilty about not being able to find time to work on this. My basic idea was to make a soft int thread that could be used to get the stuff out. Have a spin lock protected the queue at the kvprintf level. Just going to memory can just happen as it does now. Other requests would go into a buffer to be printed by the soft interrupt. I think this can get rid of all the lock ordering problems we are about to run into. Please don't let this note stop you from working on this. If you have the time go for it. I'll be glad to support anybody that has the cycles to work on this. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 22:22:46 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id B958037B502 for ; Thu, 5 Oct 2000 22:22:43 -0700 (PDT) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id WAA22455; Thu, 5 Oct 2000 22:19:31 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp02.primenet.com, id smtpdAAAFdayJR; Thu Oct 5 22:19:09 2000 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id WAA22938; Thu, 5 Oct 2000 22:22:06 -0700 (MST) From: Terry Lambert Message-Id: <200010060522.WAA22938@usr02.primenet.com> Subject: Re: Problems with kthread_exit() and SMPng To: bright@wintelcom.net (Alfred Perlstein) Date: Fri, 6 Oct 2000 05:22:06 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), mark@grondar.za (Mark Murray), jburkhol@HOME.COM (Jake Burkholder), bp@butya.kz (Boris Popov), freebsd-smp@FreeBSD.ORG In-Reply-To: <20001005151504.K27736@fw.wintelcom.net> from "Alfred Perlstein" at Oct 05, 2000 03:15:04 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > One of the options for fixing kernel logging under SMP was to have > a very simple queing mechanism to send the messages to a kernel > thread that would be repsoncible for the output. I followed that discussion when it was taking place. It makes sense that one would queue any IPC between any part of a kernel and the kernel thread that is responsible for serially processing the IPC request. So a more generic mechanism for queueing messages in the kernel is probably called for (you may rememebr my call for this back when you introduced your limited purpose API which did basically this type of queueuing). It makese sense to protect the queues with a mutex per queue, for the purposes of enqueueng and dequeueing these messages. > You see, the problem with logging is that the logging calls are in > a lot of weird places, nevermind critical ones like panic, there > needs to be a special relationship between the logging kthread and > the system which may be difficult to set up. I disagree with this completely. There does not need to be a "special relationship"; special relationships are anathema. Look at the special relationship between libkvm and kernel structures (a data interface, as you appear to be suggesting for logging), and the hell it hath wrought. As people continue to extend things rather than fixing old, broken things, this problem, and problems like it, will continue to haunt us. > You also have to take into account that the output method of > the driver might block therefore we need to hand off the > blocking to someone else. It is not permissable to block while holding a mutex; I've played this song before, and there's no need to repeat the whole thing in this thread. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 22:28:30 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 8650837B502; Thu, 5 Oct 2000 22:28:28 -0700 (PDT) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id WAA24886; Thu, 5 Oct 2000 22:25:12 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp02.primenet.com, id smtpdAAAR1a4.U; Thu Oct 5 22:23:05 2000 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id WAA22981; Thu, 5 Oct 2000 22:26:09 -0700 (MST) From: Terry Lambert Message-Id: <200010060526.WAA22981@usr02.primenet.com> Subject: Re: Problems with kthread_exit() and SMPng To: ticso@cicely5.cicely.de (Bernd Walter) Date: Fri, 6 Oct 2000 05:26:09 +0000 (GMT) Cc: jhb@FreeBSD.ORG (John Baldwin), jburkhol@home.com (Jake Burkholder), bp@butya.kz (Boris Popov), freebsd-smp@FreeBSD.ORG, mark@grondar.za (Mark Murray) In-Reply-To: <20001006002200.E40689@cicely5.cicely.de> from "Bernd Walter" at Oct 06, 2000 12:22:01 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > The situation with such functions is not uncritical as you may need to > release other mutexes before you call printf(); Holding multiple mutexes should only ever occur when doing list or other multiple pointer manipulation which needs to be done in such a way as to appear atomic, so this is pretty much a bogus argument. > Say you have a driver which is called via another driver (upcall, ...). > The other driver is not updated and calls us while holding Giant and we > need to fetch one of our own mutexes for processing. That means we have to > own Giant before we fetch our own mutex if we want to make use of printf() > or we need to release our before entering Giant to do printf(). > Otherwise we don't have a fixed mutex order :( Grab the printf mutex, queue the request, release the printf mutex. A mutex protexts data structure manipulations (in this example, the events queued for console display), _not_ code. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 22:43:42 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id 4021B37B502 for ; Thu, 5 Oct 2000 22:43:39 -0700 (PDT) Received: (qmail 2512 invoked from network); 6 Oct 2000 05:43:34 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 6 Oct 2000 05:43:34 -0000 Date: Fri, 6 Oct 2000 16:43:28 +1100 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: "Matthew N. Dodd" Cc: Mark Murray , Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 5 Oct 2000, Matthew N. Dodd wrote: > On Thu, 5 Oct 2000, Mark Murray wrote: > > There is another problem; printf's inside a kthread corrupt like > > crazy. They look very unthreadsafe. > > I've been thinking about a buffered printf. Ugh. printf has to be unbuffered so that it can be used for debugging. Anyway, printf should work in all contexts, since ddb uses it (actually db_printf) and ddb may be invoked in any context (including at inconvenient points in any locking or buffering code that you may use in an attempt to avoid printfs at inconvenient points). I believe there is no problems except in console drivers. printf itself is reentrant. The serial console driver is fairly reentrant. The syscons driver has some old reentrancy problems (it doesn't even use spltty() for the main part of the i/o, and it does lots of non-atomic updates of critical variables). These have apparently become more serious. If the problem is just interleaved output due to threads interrupting each other, then the problem is not very serious (but allowing this interruption will probably trigger the syscons bugs). printf should probably run at a higher priority than the rest of the thread. Note that it shouldn't run at a much higher priority since the console output device may be slow. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 5 23:29:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 03DDD37B66D for ; Thu, 5 Oct 2000 23:29:25 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e966T4j22994; Thu, 5 Oct 2000 23:29:04 -0700 (PDT) Date: Thu, 5 Oct 2000 23:29:04 -0700 From: Alfred Perlstein To: Terry Lambert Cc: Mark Murray , Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng Message-ID: <20001005232904.Z27736@fw.wintelcom.net> References: <20001005151504.K27736@fw.wintelcom.net> <200010060522.WAA22938@usr02.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <200010060522.WAA22938@usr02.primenet.com>; from tlambert@primenet.com on Fri, Oct 06, 2000 at 05:22:06AM +0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Terry Lambert [001005 22:22] wrote: > > One of the options for fixing kernel logging under SMP was to have > > a very simple queing mechanism to send the messages to a kernel > > thread that would be repsoncible for the output. > > I followed that discussion when it was taking place. It makes > sense that one would queue any IPC between any part of a kernel > and the kernel thread that is responsible for serially processing > the IPC request. > > So a more generic mechanism for queueing messages in the kernel > is probably called for (you may rememebr my call for this back > when you introduced your limited purpose API which did basically > this type of queueuing). What API? I honestly don't remeber introducing a message passing api into freebsd. > It makese sense to protect the queues with a mutex per queue, for > the purposes of enqueueng and dequeueing these messages. Yup. > > You see, the problem with logging is that the logging calls are in > > a lot of weird places, nevermind critical ones like panic, there > > needs to be a special relationship between the logging kthread and > > the system which may be difficult to set up. > > I disagree with this completely. There does not need to be a > "special relationship"; special relationships are anathema. Look > at the special relationship between libkvm and kernel structures > (a data interface, as you appear to be suggesting for logging), > and the hell it hath wrought. As people continue to extend things > rather than fixing old, broken things, this problem, and problems > like it, will continue to haunt us. Er, you can't have a panic stop the messaging thread from outputting the panic message, well you could but it would make debugging things a bit more difficult. > > You also have to take into account that the output method of > > the driver might block therefore we need to hand off the > > blocking to someone else. > > It is not permissable to block while holding a mutex; I've played > this song before, and there's no need to repeat the whole thing > in this thread. Yes, that's the idea, the kthread that recieves the message will do the call into the console driver so that it can block inside of it while letting the queing process just aquire a spinlock on the queue head (no long term blocking). -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 6 3:30:42 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.du.gtn.com (mail.du.gtn.com [194.77.9.57]) by hub.freebsd.org (Postfix) with ESMTP id 07BFE37B502; Fri, 6 Oct 2000 03:30:39 -0700 (PDT) Received: from mail.cicely.de (cicely.de [194.231.9.142]) by mail.du.gtn.com (8.11.0.Beta3/8.11.0.Beta3) with ESMTP id e96AUTJ17217 (using TLSv1/SSLv3 with cipher EDH-RSA-DES-CBC3-SHA (168 bits) verified OK); Fri, 6 Oct 2000 12:30:32 +0200 (MET DST) Received: from cicely5.cicely.de (cicely5.cicely.de [fec0::104:200:92ff:fe9b:20e7]) by mail.cicely.de (8.11.0.Beta1/8.11.0.Beta1) with ESMTP id e96AV0T65223; Fri, 6 Oct 2000 12:31:00 +0200 (CEST) Received: (from ticso@localhost) by cicely5.cicely.de (8.11.0/8.9.2) id e96AUr142043; Fri, 6 Oct 2000 12:30:53 +0200 (CEST) (envelope-from ticso) Date: Fri, 6 Oct 2000 12:30:53 +0200 From: Bernd Walter To: Terry Lambert Cc: John Baldwin , Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG, Mark Murray Subject: Re: Problems with kthread_exit() and SMPng Message-ID: <20001006123053.A42012@cicely5.cicely.de> References: <20001006002200.E40689@cicely5.cicely.de> <200010060526.WAA22981@usr02.primenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200010060526.WAA22981@usr02.primenet.com>; from tlambert@primenet.com on Fri, Oct 06, 2000 at 05:26:09AM +0000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, Oct 06, 2000 at 05:26:09AM +0000, Terry Lambert wrote: > > The situation with such functions is not uncritical as you may need to > > release other mutexes before you call printf(); > > Holding multiple mutexes should only ever occur when doing > list or other multiple pointer manipulation which needs to be > done in such a way as to appear atomic, so this is pretty > much a bogus argument. Which I understand as: Don't call another driver while owning a mutex because you never know what it might be waiting for. Giant is an exeption for this rule. The ugly point here is that I never thought about printf to be a call to another driver. > > Say you have a driver which is called via another driver (upcall, ...). > > The other driver is not updated and calls us while holding Giant and we > > need to fetch one of our own mutexes for processing. That means we have to > > own Giant before we fetch our own mutex if we want to make use of printf() > > or we need to release our before entering Giant to do printf(). > > Otherwise we don't have a fixed mutex order :( > > Grab the printf mutex, queue the request, release the printf > mutex. A mutex protexts data structure manipulations (in this > example, the events queued for console display), _not_ code. I agree but actually Giant is used to protect old code. -- B.Walter COSMO-Project http://www.cosmo-project.de ticso@cicely.de Usergroup info@cosmo-project.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 6 9:36:19 2000 Delivered-To: freebsd-smp@freebsd.org Received: from crewsoft.com (ns.aenet.net [157.22.214.1]) by hub.freebsd.org (Postfix) with ESMTP id 7D17937B502 for ; Fri, 6 Oct 2000 09:36:11 -0700 (PDT) Received: from [63.197.8.222] (HELO wireless-networks.com) by crewsoft.com (CommuniGate Pro SMTP 3.3b5) with ESMTP id 319753; Fri, 06 Oct 2000 09:38:03 -0700 Message-ID: <39DDFF70.E09EB724@wireless-networks.com> Date: Fri, 06 Oct 2000 09:36:00 -0700 From: Cedric Berger X-Mailer: Mozilla 4.73 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Bruce Evans Cc: "Matthew N. Dodd" , Mark Murray , Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Bruce Evans wrote: > > On Thu, 5 Oct 2000, Matthew N. Dodd wrote: > > > On Thu, 5 Oct 2000, Mark Murray wrote: > > > There is another problem; printf's inside a kthread corrupt like > > > crazy. They look very unthreadsafe. > > > > I've been thinking about a buffered printf. > > Ugh. printf has to be unbuffered so that it can be used for debugging. I'm not at all aware of how printf works on FreeBSD, but what about a "half-buffered" printf as it is found in most RTOS (i.e. the whole printf() message is buffered, with only a single I/O write() call at the end) ? Cedric To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 6 9:52:45 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 180D937B502 for ; Fri, 6 Oct 2000 09:52:43 -0700 (PDT) Received: from berserker.bsdi.com (cp@localhost [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id KAA14529 for ; Fri, 6 Oct 2000 10:52:38 -0600 (MDT) Message-Id: <200010061652.KAA14529@berserker.bsdi.com> To: freebsd-smp@freebsd.org Subject: Kernel printf Was: Problems with kthread_exit() and SMPng In-reply-to: Your message of "Fri, 06 Oct 2000 12:30:53 +0200." <20001006123053.A42012@cicely5.cicely.de> From: Chuck Paterson Date: Fri, 06 Oct 2000 10:52:38 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org There is no single mutex you can grab and be safe. If the printf is coming from some place with a spin lock you can't grab a sleep lock to protect. You can't hold a spin lock while you call into the output routines. Furthermore you really don't know that you won't introduce a lock ordering problem. Just grabbing Giant isn't even safe in the short term because there can be lock ordering problems. If you queue and then do the output with a soft interrupt you should get output immediately. The cases were you don't is from the bottom half, when a mutex is held someplace else that prevents the output from occurring, or the current thread has its effective priority raised for some reason, such as holding a spin lock. This would seem to meet all the real needs and is only a minor PITA to implement. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 6 10:39:31 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sasami.jurai.net (sasami.jurai.net [63.67.141.99]) by hub.freebsd.org (Postfix) with ESMTP id 76C3337B66C for ; Fri, 6 Oct 2000 10:39:29 -0700 (PDT) Received: from localhost (winter@localhost) by sasami.jurai.net (8.9.3/8.8.7) with ESMTP id NAA13947; Fri, 6 Oct 2000 13:38:50 -0400 (EDT) Date: Fri, 6 Oct 2000 13:38:50 -0400 (EDT) From: "Matthew N. Dodd" To: Cedric Berger Cc: Bruce Evans , Mark Murray , Jake Burkholder , Boris Popov , freebsd-smp@FreeBSD.ORG Subject: Re: Problems with kthread_exit() and SMPng In-Reply-To: <39DDFF70.E09EB724@wireless-networks.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 6 Oct 2000, Cedric Berger wrote: > I'm not at all aware of how printf works on FreeBSD, but what about a > "half-buffered" printf as it is found in most RTOS (i.e. the whole > printf() message is buffered, with only a single I/O write() call at > the end) ? Thats kind of what I was looking for in a 'bprintf()' or something... Output on EOL or len > wraplen. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | This Space For Rent | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 6 12:37:20 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id E4D6137B503; Fri, 6 Oct 2000 12:37:15 -0700 (PDT) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id MAA27363; Fri, 6 Oct 2000 12:37:04 -0700 (MST) Received: from usr05.primenet.com(206.165.6.205) via SMTP by smtp05.primenet.com, id smtpdAAALHaGK0; Fri Oct 6 12:34:50 2000 Received: (from tlambert@localhost) by usr05.primenet.com (8.8.5/8.8.5) id MAA20583; Fri, 6 Oct 2000 12:34:19 -0700 (MST) From: Terry Lambert Message-Id: <200010061934.MAA20583@usr05.primenet.com> Subject: Re: Problems with kthread_exit() and SMPng To: ticso@cicely5.cicely.de (Bernd Walter) Date: Fri, 6 Oct 2000 19:34:14 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), jhb@FreeBSD.ORG (John Baldwin), jburkhol@home.com (Jake Burkholder), bp@butya.kz (Boris Popov), freebsd-smp@FreeBSD.ORG, mark@grondar.za (Mark Murray) In-Reply-To: <20001006123053.A42012@cicely5.cicely.de> from "Bernd Walter" at Oct 06, 2000 12:30:53 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Holding multiple mutexes should only ever occur when doing > > list or other multiple pointer manipulation which needs to be > > done in such a way as to appear atomic, so this is pretty > > much a bogus argument. > > Which I understand as: > Don't call another driver while owning a mutex because you never > know what it might be waiting for. I'd say "don't call a function holding a mutex", but would grudgingly tolerate functions whose only purpose was a common data manipulation -- though I'd point out that they would be better done either as inlines or as macros instead (e.g. NDINIT()). > Giant is an exeption for this rule. It's orthogonal. People want to keep the giant lock, and they want to use mutexes; the two are not in the same collision domain. I would pretty much say "never do this" or "do this at your peril, and if you accept the peril, then document it for others". > The ugly point here is that I never thought about printf to be a call > to another driver. Any driver that calls printf is going to end up calling the console driver, sooner or later. The way to deal with this is really a mutex and a condition variable, since debugging really wants to wait until the console has displayed the message before going on to code that could crash the machine. For things you don't care about, you could have either a different call (an async printf) or a flag you pass as one of the arguments. Debugging flags would override this and synchronize the I/O despite the flag/entry point, by making the caller wait on the condition variable, which would indicate that the message had been successfully written to the console. It's pretty trivial to implement this, even with multiple callers, since you would make the condition variable be monotonically increasing, and wait on the queued even order since boot, so that multiple callers could wait simultaneously and be awakened in FIFO order. This would mean another counter to track the queue head next order assignment number, since otherwise, you would only be able to support a depth of one. This would be incremented automatically by the caller while the mutex was held, giving the value that the condition variable will have after the message has been output, and the caller should continue (increments occur in both the sync and async cases, of course; in the async case, it merely doesn't wait for the condition variable to catch up). The code itself would change the printf call to XDR the arguments to printf into queue elements, mutex protect the queue, do the insert, release the mutex, and return (async case; the sync case would want an atomic release the mutex and wait on condition variable). The kernel thread which would take over responsibility for displaying data on the console would acquire the mutex, dequeue an entry, release the mutex, process the output to completion, acquire the mutex, increment and trigger the condition variable, release the mutex. Pretty trivial, and no recursion problems and no giant lock problems. The only remaining implementation details to decide would be whether the caller allocated and printf free the queue elements from/to main memory or a free list, if a free list would be fixed size or grow, and what to do with a fixed size freelist when the list was empty and a caller attempted to queue a display request. > > Grab the printf mutex, queue the request, release the printf > > mutex. A mutex protexts data structure manipulations (in this > > example, the events queued for console display), _not_ code. > > I agree but actually Giant is used to protect old code. Irrelevent, I think, if he only legitimate entrypoint for displaying console data was the call queue. As I said previously, this would be very useful going forward, if call queues could be generalized so that everyone could use them. It seems to me that they would be particularly useful for communication between top and bottom drivers, and similar situations. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 6 18:10:37 2000 Delivered-To: freebsd-smp@freebsd.org Received: from relay.butya.kz (butya-gw.butya.kz [212.154.129.94]) by hub.freebsd.org (Postfix) with ESMTP id E4D7937B503; Fri, 6 Oct 2000 18:10:32 -0700 (PDT) Received: by relay.butya.kz (Postfix, from userid 1000) id EF85728AAF; Sat, 7 Oct 2000 08:10:27 +0700 (ALMST) Received: from localhost (localhost [127.0.0.1]) by relay.butya.kz (Postfix) with ESMTP id E7DA328645; Sat, 7 Oct 2000 08:10:27 +0700 (ALMST) Date: Sat, 7 Oct 2000 08:10:27 +0700 (ALMST) From: Boris Popov To: John Baldwin Cc: freebsd-smp@FreeBSD.org Subject: RE: Problems with kthread_exit() and SMPng In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 6 Oct 2000, Boris Popov wrote: > > It should have died much earlier if you had INVARIANTS turned on. :( It > > looks like you are releasing a mutex you probably do not own because > > cpu_exit() (called by exit1() -> exit() -> kthread_exit()) releases Giant > > as one of its final tasks. > > This is probably the bug somewhere in the diagnostic code. I have > INVARIANTS/INVARIANT_SUPPORT/DIAGNOSTIC turned on and UP machine just > panics in the mtx_exit_hard() while SMP machine silently reboots :( Actually, this is happens because #ifdef INVARIANTS is hidden behind #ifdef SMP_DEBUG which looks slightly misleading. Eg: if one decides to compile kernel with INVARIANTS then he expects sanity checks in the all core subsystems. -- Boris Popov http://www.butya.kz/~bp/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 6 18:19:45 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pike.osd.bsdi.com (pike.osd.bsdi.com [204.216.28.222]) by hub.freebsd.org (Postfix) with ESMTP id 7D8DB37B502 for ; Fri, 6 Oct 2000 18:19:43 -0700 (PDT) Received: from laptop.baldwin.cx (ether.osd.bsdi.com [204.216.28.196]) by pike.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e971JLi59549; Fri, 6 Oct 2000 18:19:21 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: Date: Fri, 06 Oct 2000 18:19:39 -0700 (PDT) From: John Baldwin To: Boris Popov Subject: RE: Problems with kthread_exit() and SMPng Cc: freebsd-smp@FreeBSD.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 07-Oct-00 Boris Popov wrote: > On Fri, 6 Oct 2000, Boris Popov wrote: > >> > It should have died much earlier if you had INVARIANTS turned on. :( It >> > looks like you are releasing a mutex you probably do not own because >> > cpu_exit() (called by exit1() -> exit() -> kthread_exit()) releases Giant >> > as one of its final tasks. >> >> This is probably the bug somewhere in the diagnostic code. I have >> INVARIANTS/INVARIANT_SUPPORT/DIAGNOSTIC turned on and UP machine just >> panics in the mtx_exit_hard() while SMP machine silently reboots :( > > Actually, this is happens because #ifdef INVARIANTS is hidden > behind #ifdef SMP_DEBUG which looks slightly misleading. Eg: if one > decides to compile kernel with INVARIANTS then he expects sanity checks in > the all core subsystems. Probably in the future, we can fold SMP_DEBUG into INVARIANTS, with some parts under a seperate MUTEX_DEBUG (as that is a better name). Jason, feel free to stick that as a small line-item on the todo list. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message