From owner-freebsd-smp Sun Sep 24 11:27:16 2000 Delivered-To: freebsd-smp@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 7E66137B42C; Sun, 24 Sep 2000 11:27:07 -0700 (PDT) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id UAA49313; Sun, 24 Sep 2000 20:26:59 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Warner Losh Cc: The Hermit Hacker , Bruce Evans , freebsd-current@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG Subject: Re: 'interrupt-level buffer overflows' for sio device? References: <200009080314.VAA50701@harmony.village.org> From: Dag-Erling Smorgrav Date: 24 Sep 2000 20:26:58 +0200 In-Reply-To: Warner Losh's message of "Thu, 07 Sep 2000 21:14:58 -0600" Message-ID: Lines: 26 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Warner Losh writes: > In message The Hermit Hacker writes: > : Okay, I'm a little confused here ... from what I'm reading/following, this > : isn't a new problem ... or is it? If not, why has it suddenly manifested > : itself with the new SMP code? > It seems to be new with the SMP code. At least that's what my reading > of the original message is. I'm seeing the same thing as Marc, but I have a little more info: 1) it started after the SMPng commit. 2) it's not just sio, it's everything: - interactive response is markedly slower. - the keyboard autorepeats slower than it used to (even with kbdcontrol -r fast). - I/O bound processes such as mpg123 or cvsup will pause briefly when there is other I/O going on (moving the mouse, typing fast, holding down a key on the keyboard), which they didn't use to. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Sep 24 11:30:28 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 8A73E37B422; Sun, 24 Sep 2000 11:30:20 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8OIU8e29105; Sun, 24 Sep 2000 11:30:08 -0700 (PDT) Date: Sun, 24 Sep 2000 11:30:08 -0700 From: Alfred Perlstein To: Dag-Erling Smorgrav Cc: Warner Losh , The Hermit Hacker , Bruce Evans , freebsd-current@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG Subject: Re: 'interrupt-level buffer overflows' for sio device? Message-ID: <20000924113008.N9141@fw.wintelcom.net> References: <200009080314.VAA50701@harmony.village.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from des@ofug.org on Sun, Sep 24, 2000 at 08:26:58PM +0200 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Dag-Erling Smorgrav [000924 11:27] wrote: > Warner Losh writes: > > In message The Hermit Hacker writes: > > : Okay, I'm a little confused here ... from what I'm reading/following, this > > : isn't a new problem ... or is it? If not, why has it suddenly manifested > > : itself with the new SMP code? > > It seems to be new with the SMP code. At least that's what my reading > > of the original message is. > > I'm seeing the same thing as Marc, but I have a little more info: > > 1) it started after the SMPng commit. > > 2) it's not just sio, it's everything: > > - interactive response is markedly slower. > > - the keyboard autorepeats slower than it used to (even with > kbdcontrol -r fast). > > - I/O bound processes such as mpg123 or cvsup will pause briefly > when there is other I/O going on (moving the mouse, typing fast, > holding down a key on the keyboard), which they didn't use to. This is basically a result of the entire kernel running at the equivelant of splhigh, all interrupts are blocked until a context switch in kernel land. There's work in progress to mpsafe the drivers (at least for ethernet, more will arrive later). -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Sep 24 11:39:23 2000 Delivered-To: freebsd-smp@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id EEC1937B422; Sun, 24 Sep 2000 11:39:18 -0700 (PDT) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id UAA49387; Sun, 24 Sep 2000 20:39:14 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Alfred Perlstein Cc: Warner Losh , The Hermit Hacker , Bruce Evans , freebsd-current@FreeBSD.ORG, freebsd-smp@FreeBSD.ORG Subject: Re: 'interrupt-level buffer overflows' for sio device? References: <200009080314.VAA50701@harmony.village.org> <20000924113008.N9141@fw.wintelcom.net> From: Dag-Erling Smorgrav Date: 24 Sep 2000 20:39:14 +0200 In-Reply-To: Alfred Perlstein's message of "Sun, 24 Sep 2000 11:30:08 -0700" Message-ID: Lines: 14 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Alfred Perlstein writes: > This is basically a result of the entire kernel running at the > equivelant of splhigh, all interrupts are blocked until a context > switch in kernel land. > > There's work in progress to mpsafe the drivers (at least for > ethernet, more will arrive later). OK. Assuming I wanted to try and help with this work, where would be a good place to start finding out what needs to be done? DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Sep 24 12:14:17 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id B0BCE37B422 for ; Sun, 24 Sep 2000 12:14:10 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8OJE0i00206; Sun, 24 Sep 2000 12:14:00 -0700 (PDT) Date: Sun, 24 Sep 2000 12:14:00 -0700 From: Alfred Perlstein To: Dag-Erling Smorgrav Cc: Warner Losh , The Hermit Hacker , Bruce Evans , freebsd-smp@FreeBSD.ORG Subject: Re: 'interrupt-level buffer overflows' for sio device? Message-ID: <20000924121359.O9141@fw.wintelcom.net> References: <200009080314.VAA50701@harmony.village.org> <20000924113008.N9141@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: ; from des@ofug.org on Sun, Sep 24, 2000 at 08:39:14PM +0200 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Dag-Erling Smorgrav [000924 11:39] wrote: > Alfred Perlstein writes: > > This is basically a result of the entire kernel running at the > > equivelant of splhigh, all interrupts are blocked until a context > > switch in kernel land. > > > > There's work in progress to mpsafe the drivers (at least for > > ethernet, more will arrive later). > > OK. Assuming I wanted to try and help with this work, where would be a > good place to start finding out what needs to be done? Have a look at what was recently done to the fxp driver, basically protect the instance of the driver from itself, ie, if you see splfoo() in the driver, identify what it's trying to protect against and figure where the mutex needs to be. You want to block things like ioctls and user entry points from interfering with the interrupt context of the driver. ATM John Baldwin is working on making some sort of flag for the newbus interrupt and cdev initilization routines so that a driver can mark itself mpsafe. Other things need to be taken care of though, fxp can't have Giant (the equivelant of splhigh) removed until several things happen: ether_input/output is made mpsafe (trivial) the ifqueue structs have mutexes put into them (trivial) the mbuf subsystem made made mpsafe (done, but not committed) malloc made mpsafe (done and committed) softinterrupt scheduling is made mpsafe (I think this is on John's plate) -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Sep 24 12:20:17 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wintelcom.net (ns1.wintelcom.net [209.1.153.20]) by hub.freebsd.org (Postfix) with ESMTP id 8A85037B422 for ; Sun, 24 Sep 2000 12:20:11 -0700 (PDT) Received: (from bright@localhost) by fw.wintelcom.net (8.10.0/8.10.0) id e8OJK1500402; Sun, 24 Sep 2000 12:20:01 -0700 (PDT) Date: Sun, 24 Sep 2000 12:20:01 -0700 From: Alfred Perlstein To: Dag-Erling Smorgrav Cc: Warner Losh , The Hermit Hacker , Bruce Evans , freebsd-smp@FreeBSD.ORG Subject: Re: 'interrupt-level buffer overflows' for sio device? Message-ID: <20000924122000.P9141@fw.wintelcom.net> References: <200009080314.VAA50701@harmony.village.org> <20000924113008.N9141@fw.wintelcom.net> <20000924121359.O9141@fw.wintelcom.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.4i In-Reply-To: <20000924121359.O9141@fw.wintelcom.net>; from bright@wintelcom.net on Sun, Sep 24, 2000 at 12:14:00PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org * Alfred Perlstein [000924 12:14] wrote: > * Dag-Erling Smorgrav [000924 11:39] wrote: > > Alfred Perlstein writes: > > > This is basically a result of the entire kernel running at the > > > equivelant of splhigh, all interrupts are blocked until a context > > > switch in kernel land. > > > > > > There's work in progress to mpsafe the drivers (at least for > > > ethernet, more will arrive later). > > > > OK. Assuming I wanted to try and help with this work, where would be a > > good place to start finding out what needs to be done? > Er, let me follow up in order to clarify a few things... > Have a look at what was recently done to the fxp driver, basically > protect the instance of the driver from itself, ie, if you see > splfoo() in the driver, identify what it's trying to protect against > and figure where the mutex needs to be. This basically means that any time you see spl, you'll want to see what exactly the spl is protecting and use a mutex in that structure or subsystem. Also, all the instances of locking using mutual exclusion need to be replaced with msleep interlocking against a conditional variable. For a detailed explanation of such things you might want to have a look at: http://people.freebsd.org/~jasone/smp/ http://people.freebsd.org/~alfred/mpsafe/ http://www.technokratis.com/code/mbuf/mtx_journal I'm hoping you're going to be at BSDcon. -- -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] "I have the heart of a child; I keep it in a jar on my desk." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Sep 24 13:24: 0 2000 Delivered-To: freebsd-smp@freebsd.org Received: from flood.ping.uio.no (flood.ping.uio.no [129.240.78.31]) by hub.freebsd.org (Postfix) with ESMTP id 4032037B422 for ; Sun, 24 Sep 2000 13:23:54 -0700 (PDT) Received: (from des@localhost) by flood.ping.uio.no (8.9.3/8.9.3) id WAA49698; Sun, 24 Sep 2000 22:23:35 +0200 (CEST) (envelope-from des@ofug.org) X-URL: http://www.ofug.org/~des/ X-Disclaimer: The views expressed in this message do not necessarily coincide with those of any organisation or company with which I am or have been affiliated. To: Alfred Perlstein Cc: Warner Losh , The Hermit Hacker , Bruce Evans , freebsd-smp@FreeBSD.ORG Subject: Re: 'interrupt-level buffer overflows' for sio device? References: <200009080314.VAA50701@harmony.village.org> <20000924113008.N9141@fw.wintelcom.net> <20000924121359.O9141@fw.wintelcom.net> <20000924122000.P9141@fw.wintelcom.net> From: Dag-Erling Smorgrav Date: 24 Sep 2000 22:23:35 +0200 In-Reply-To: Alfred Perlstein's message of "Sun, 24 Sep 2000 12:20:01 -0700" Message-ID: Lines: 15 User-Agent: Gnus/5.0802 (Gnus v5.8.2) Emacs/20.4 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Alfred Perlstein writes: > * Alfred Perlstein [000924 12:14] wrote: > [...] Thanks for the info! > I'm hoping you're going to be at BSDcon. Unfortunately, no. I'm in the process of starting a company, which means I'm relatively low on money and time; and on top of that, my mother's sixtieth birthday is on October 22nd. DES -- Dag-Erling Smorgrav - des@ofug.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Sep 24 14: 5:54 2000 Delivered-To: freebsd-smp@freebsd.org Received: from server.baldwin.cx (server.geekhouse.net [64.81.6.52]) by hub.freebsd.org (Postfix) with ESMTP id 3145837B424 for ; Sun, 24 Sep 2000 14:05:47 -0700 (PDT) Received: from john.baldwin.cx (root@john.baldwin.cx [192.168.1.18]) by server.baldwin.cx (8.9.3/8.9.3) with ESMTP id OAA95723; Sun, 24 Sep 2000 14:07:37 -0700 (PDT) (envelope-from john@baldwin.cx) Received: (from john@localhost) by john.baldwin.cx (8.9.3/8.9.3) id OAA53604; Sun, 24 Sep 2000 14:06:53 -0700 (PDT) (envelope-from john) Message-Id: <200009242106.OAA53604@john.baldwin.cx> X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <20000924121359.O9141@fw.wintelcom.net> Date: Sun, 24 Sep 2000 14:06:53 -0700 (PDT) From: John Baldwin To: Alfred Perlstein Subject: Re: 'interrupt-level buffer overflows' for sio device? Cc: freebsd-smp@FreeBSD.ORG, Bruce Evans , The Hermit Hacker , Warner Losh , Dag-Erling Smorgrav Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 24-Sep-00 Alfred Perlstein wrote: > * Dag-Erling Smorgrav [000924 11:39] wrote: >> Alfred Perlstein writes: >> > This is basically a result of the entire kernel running at the >> > equivelant of splhigh, all interrupts are blocked until a context >> > switch in kernel land. >> > >> > There's work in progress to mpsafe the drivers (at least for >> > ethernet, more will arrive later). >> >> OK. Assuming I wanted to try and help with this work, where would be a >> good place to start finding out what needs to be done? > > Have a look at what was recently done to the fxp driver, basically > protect the instance of the driver from itself, ie, if you see > splfoo() in the driver, identify what it's trying to protect against > and figure where the mutex needs to be. > > You want to block things like ioctls and user entry points from > interfering with the interrupt context of the driver. > > ATM John Baldwin is working on making some sort of flag for the > newbus interrupt and cdev initilization routines so that a driver > can mark itself mpsafe. Only for cdevsw. newbus interrupt handlers already have the flag, and have had it for a while. > Other things need to be taken care of though, fxp can't have Giant > (the equivelant of splhigh) removed until several things happen: > > ether_input/output is made mpsafe (trivial) > the ifqueue structs have mutexes put into them (trivial) > the mbuf subsystem made made mpsafe (done, but not committed) > malloc made mpsafe (done and committed) > softinterrupt scheduling is made mpsafe (I think this is on John's plate) Err, this should be mpsafe on the x86 already. It will be mpsafe on the alpha shortly. -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Sep 24 21:23: 0 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mustang.nitrous.net (nitrous.net [63.78.15.50]) by hub.freebsd.org (Postfix) with ESMTP id 44C1037B424 for ; Sun, 24 Sep 2000 21:22:30 -0700 (PDT) Received: (from denver@localhost) by mustang.nitrous.net (8.10.0/w00t-1.2) id e8P3vOK15621 for freebsd-smp@freebsd.org; Sun, 24 Sep 2000 20:57:24 -0700 (MST) Date: Sun, 24 Sep 2000 20:57:23 -0700 From: Denver Maddux To: freebsd-smp@freebsd.org Subject: SMP on a Netfinity 6000R Message-ID: <20000924205723.G10774@nitrous.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.7i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi. I've recently been purchased some Netfinity 6000Rs and am wanting to run FreeBSD on them. I've loaded 4.1-CURRENT on one of them with a single processor, and it works like a champ (fast as hell). My problem is... I have 3 other processors for these things, as the hold a total of 4. I decided to do my initial load with only 2 processors per box just to test and get familiar with them (no need to unpack what I may not be able to use). I've enabled SMP in the kernel config, and at boot time all goes well until the 2nd processor is loaded. DDB shows it halt after loading: db> n After 2 instructions (0 loads, 0 stores), Stopped at atkbd_isa_intr+0x19: ret db> n After 248 instructions (0 loads, 0 stores), Stopped at doreti_iret: iret db> n ....... And the kernel halts. Booting with a GENERIC kernel shows that the next device to load is the pass1 device... this never happens with SMP enabled and the last message seen before halting is "Waiting 15 seconds for SCSI devices to settle". I've seen in the archives that alot of people have used these boxes in the past successfully with an SMP kernel, but I can't seem to find anything specific to the 6000R model. Currently, my kernel config is the GENERIC kernel with SMP enabled. It made the most sense to me since is the first time I've used one of these boxes, and the GENERIC kernel worked just fine. My SMP configs are: options SMP options APIC_IO options NCPU=2 options NBUS=10 options NAPIC=2 options NINTR=39 Below are output from dmesg with the generic kernel and mptable output. Does anyone have any ideas? Thanks, Denver ---------------------------------------------------------------- Copyright (c) 1992-2000 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.1-RELEASE #0: Fri Jul 28 14:30:31 GMT 2000 jkh@ref4.freebsd.org:/usr/src/sys/compile/GENERIC Timecounter "i8254" frequency 1193182 Hz CPU: Unknown 80686 (701.62-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x6a1 Stepping = 1 Features=0x383fbff real memory = 1073713152 (1048548K bytes) avail memory = 1040621568 (1016232K bytes) Pentium Pro MTRR support enabled md0: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib2: on motherboard pci2: on pcib2 ahc0: port 0x4000-0x40ff mem 0xefbff000-0xefbfffff irq 10 at device 1.0 on pci2 ahc0: aic7899 Wide Channel A, SCSI Id=7, 16/255 SCBs ahc1: port 0x4100-0x41ff mem 0xefbfe000-0xefbfefff irq 10 at device 1.1 on pci2 ahc1: aic7899 Wide Channel B, SCSI Id=7, 16/255 SCBs pcib0: on motherboard pci0: on pcib0 fxp0: port 0x2200-0x223f mem 0xfea00000-0xfeafffff,0xfebff000-0xfebfffff irq 11 at device 1.0 on pci0 fxp0: Ethernet address 00:06:29:8f:29:c2 lnc0: port 0x2240-0x225f mem 0xfebfec00-0xfebfec1f irq 11 at device 5.0 on pci0 lnc0: driver is using old-style compatability shims pci0: at 6.0 isab0: at device 15.0 on pci0 isa0: on isab0 atapci0: port 0x700-0x70f at device 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ohci0: mem 0xfebfd000-0xfebfdfff irq 15 at device 15.2 on pci0 usb0: OHCI version 1.0, legacy support usb0: on ohci0 usb0: USB revision 1.0 uhub0: (unknown) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered pcib3: on motherboard pci3: on pcib3 pcib5: on motherboard pci5: on pcib5 fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppi0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port plip0: on ppbus0 acd0: CDROM at ata0-master using PIO4 Waiting 15 seconds for SCSI devices to settle pass1 at ahc1 bus 0 target 8 lun 0 pass1: Fixed Processor SCSI-2 device pass1: 3.300MB/s transfers da0 at ahc1 bus 0 target 2 lun 0 da0: Fixed Direct Access SCSI-3 device da0: 80.000MB/s transfers (40.000MHz, offset 31, 16bit), Tagged Queueing Enabled da0: 17357MB (35548320 512 byte sectors: 255H 63S/T 2212C) Mounting root from ufs:da0s1a =============================================================================== MPTable, version 2.0.15 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: EBDA physical address: 0x0009e1d0 signature: '_MP_' length: 16 bytes version: 1.4 checksum: 0xd6 mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x0009e1e0 signature: 'PCMP' base table length: 508 version: 1.4 checksum: 0x04 OEM ID: 'UNISYS ' Product ID: ' ' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 55 local APIC address: 0xfee00000 extended table length: 228 extended table checksum: 158 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 3 0x11 BSP, usable 6 10 1 0x0301 0 0x11 AP, usable 6 10 1 0x0301 -- Bus: Bus ID Type 0 PCI 1 PCI 2 PCI 3 PCI 4 PCI 5 PCI 6 PCI 7 PCI 8 PCI 9 ISA -- I/O APICs: APIC ID Version State Address 14 0x11 usable 0xfec00000 13 0x11 usable 0xfec01000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# INT conforms conforms 9 1 14 1 INT conforms conforms 9 0 14 2 INT conforms conforms 9 3 14 3 INT conforms conforms 9 4 14 4 INT active-lo level 9 5 14 5 INT conforms conforms 9 6 14 6 INT conforms conforms 9 7 14 7 INT conforms conforms 9 8 14 8 INT conforms conforms 9 12 14 12 INT conforms conforms 9 13 14 13 INT conforms conforms 9 14 14 14 INT conforms conforms 0 1:A 13 4 INT conforms conforms 0 5:A 13 0 INT conforms conforms 0 15:A 13 3 INT conforms conforms 2 1:A 13 1 INT conforms conforms 2 1:B 13 2 INT conforms conforms 0 1:B 14 15 INT conforms conforms 0 1:C 14 9 INT conforms conforms 0 1:D 14 9 INT conforms conforms 5 2:A 13 5 INT conforms conforms 5 2:B 14 9 INT conforms conforms 5 2:C 14 9 INT conforms conforms 5 2:D 14 9 INT conforms conforms 5 3:A 13 6 INT conforms conforms 5 3:B 14 10 INT conforms conforms 5 3:C 14 9 INT conforms conforms 5 3:D 14 9 INT conforms conforms 5 4:A 13 7 INT conforms conforms 5 4:B 14 11 INT conforms conforms 5 4:C 14 9 INT conforms conforms 5 4:D 14 9 INT conforms conforms 2 5:A 13 8 INT conforms conforms 2 5:B 14 15 INT conforms conforms 2 5:C 14 9 INT conforms conforms 2 5:D 14 9 INT conforms conforms 2 6:A 13 9 INT conforms conforms 2 6:B 14 9 INT conforms conforms 2 6:C 14 9 INT conforms conforms 2 6:D 14 9 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# NMI conforms conforms 9 0 255 1 ExtINT conforms conforms 9 0 255 0 ------------------------------------------------------------------------------- MP Config Extended Table Entries: -- bus ID: 0 address type: memory address address base: 0xa0000 address range: 0x20000 -- bus ID: 0 address type: memory address address base: 0xc0000 address range: 0x20000 -- bus ID: 0 address type: prefetch address address base: 0xefc00000 address range: 0x8a00000 -- bus ID: 0 address type: memory address address base: 0xf8600000 address range: 0x7a00000 -- bus ID: 2 address type: prefetch address address base: 0xed000000 address range: 0x800000 -- bus ID: 2 address type: memory address address base: 0xed800000 address range: 0x2400000 -- bus ID: 5 address type: memory address address base: 0xeb800000 address range: 0xc00000 -- bus ID: 5 address type: prefetch address address base: 0xec400000 address range: 0xc00000 -- bus ID: 0 address type: I/O address address base: 0x0 address range: 0x4000 -- bus ID: 2 address type: I/O address address base: 0x4000 address range: 0x3000 -- bus ID: 5 address type: I/O address address base: 0x7000 address range: 0x9000 -- bus ID: 9 bus info: 0x01 parent bus ID: 0 ------------------------------------------------------------------------------- # SMP kernel config file options: # Required: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optional (built-in defaults will work in most cases): #options NCPU=2 # number of CPUs #options NBUS=10 # number of busses #options NAPIC=2 # number of IO APICs #options NINTR=39 # number of INTs =============================================================================== To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Sep 25 10:14:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from ipamzlx.physik.uni-mainz.de (ipamzlx.Physik.Uni-Mainz.DE [134.93.180.54]) by hub.freebsd.org (Postfix) with ESMTP id 1E74137B424; Mon, 25 Sep 2000 10:14:37 -0700 (PDT) Received: from ipamzlx.Physik.Uni-Mainz.DE (ipamzlx.Physik.Uni-Mainz.DE [134.93.180.54]) by ipamzlx.physik.uni-mainz.de (8.11.0/8.9.3) with ESMTP id e8PHFnp93386; Mon, 25 Sep 2000 19:15:49 +0200 (CEST) (envelope-from ohartman@ipamzlx.physik.uni-mainz.de) Date: Mon, 25 Sep 2000 19:15:49 +0200 (CEST) From: "O. Hartmann" To: freebsd-smp@freebsd.org Cc: freebsd-hardware@freebsd.org Subject: TYAN Thunder 2500/MP Table Error Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Dear Sirs. I run successfully a TYAN Thunder 2500 mainboard with FreeBSD 4.1-STABLE, BIOS Beta 1.03/896 for the LSI 53C896 based SCSI controler. When booting, I obtain a lot of SMP messages from the kernel, but at one stage, I receive this error: APIC_IO: Testing 8254 interrupt delivery APIC_IO: Broken MP table detected: 8254 is not connected to IOAPIC #0 intpin 2 APIC_IO: routing 8254 via 8259 and IOAPIC #0 intpin 0 SMP works fine, but it seems that there is something wron with the MP table. Is this a serious error message or is it simply a warning? What kind of performance penalty does it results in? And: hopefully, is there a chance to get rid of it with a new BIOS update in the future? Thanks in advance, Gruss O. Hartmann ------------------------------------------------------------------- ohartman@ipamzlx.physik.uni-mainz.de Klimadatenserver des IPA, Universitaet Mainz Netzwerk- und Systembetreuung To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Sep 25 12:32:42 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mass.osd.bsdi.com (adsl-63-206-89-203.dsl.snfc21.pacbell.net [63.206.89.203]) by hub.freebsd.org (Postfix) with ESMTP id 1FC8737B50C; Mon, 25 Sep 2000 12:31:54 -0700 (PDT) Received: from mass.osd.bsdi.com (localhost [127.0.0.1]) by mass.osd.bsdi.com (8.11.0/8.9.3) with ESMTP id e8PJVKA13966; Mon, 25 Sep 2000 12:31:21 -0700 (PDT) (envelope-from msmith@mass.osd.bsdi.com) Message-Id: <200009251931.e8PJVKA13966@mass.osd.bsdi.com> X-Mailer: exmh version 2.1.1 10/15/1999 To: "O. Hartmann" Cc: freebsd-smp@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: TYAN Thunder 2500/MP Table Error In-reply-to: Your message of "Mon, 25 Sep 2000 19:15:49 +0200." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 25 Sep 2000 12:31:20 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This is an informational message only, and you should ignore it. > Dear Sirs. > I run successfully a TYAN Thunder 2500 mainboard with FreeBSD 4.1-STABLE, BIOS Beta > 1.03/896 for the LSI 53C896 based SCSI controler. When booting, I obtain a lot of > SMP messages from the kernel, but at one stage, I receive this error: > > APIC_IO: Testing 8254 interrupt delivery > APIC_IO: Broken MP table detected: 8254 is not connected to IOAPIC #0 intpin 2 > APIC_IO: routing 8254 via 8259 and IOAPIC #0 intpin 0 > > SMP works fine, but it seems that there is something wron with the MP table. > Is this a serious error message or is it simply a warning? What kind of performance penalty does > it results in? And: hopefully, is there a chance to get rid of it with a new > BIOS update in the future? > > Thanks in advance, > > Gruss O. Hartmann > ------------------------------------------------------------------- > ohartman@ipamzlx.physik.uni-mainz.de > > Klimadatenserver des IPA, Universitaet Mainz > Netzwerk- und Systembetreuung > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Sep 25 15:18:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mustang.nitrous.net (nitrous.net [63.78.15.50]) by hub.freebsd.org (Postfix) with ESMTP id 6BCF237B424 for ; Mon, 25 Sep 2000 15:18:09 -0700 (PDT) Received: (from denver@localhost) by mustang.nitrous.net (8.10.0/w00t-1.2) id e8PMI0831079 for freebsd-smp@FreeBSD.ORG; Mon, 25 Sep 2000 15:18:00 -0700 (MST) Date: Mon, 25 Sep 2000 15:18:00 -0700 From: Denver Maddux To: freebsd-smp@FreeBSD.ORG Subject: Re: SMP on a Netfinity 6000R Message-ID: <20000925151759.A17149@nitrous.net> References: <20000924205723.G10774@nitrous.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.7i In-Reply-To: <20000924205723.G10774@nitrous.net> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org To follow up, I've been able to obtain some more debug info. I'm now able to see this output while breaking into debug as the kernel loads (boot -dv kernel). Fatal trap 12: page fault while in kernel mode mp_lock = 01000002; cpuid = 1; lapic.id = 00000000 fault virtual address = 0cff80e000 fault code = supervisor read, page not present instruction pointer = 0x8:0xc0226a3e stack pointer = 0x10:0xff80df5c frame pointer = 0x10:0xff80df68 code segment = base 0x0, limit 0xfffff, type 0x1b = DPC 0, pres 1, def23 1, gran 1 processor eflags = trace trap, interrupt enabled, resume, IOPL = 0 current process = Idle interrupt mask = none <- SMP: XXX kernel: type 12 trap, code=0 My SMP configs are now: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O options NBUS=10 # number of busses options NAPIC=2 # number of IO APICs What does that mean above? Thanks, Denver Denver Maddux wrote: > Hi. I've recently been purchased some Netfinity 6000Rs and am wanting to run > FreeBSD on them. I've loaded 4.1-CURRENT on one of them with a single > processor, and it works like a champ (fast as hell). My problem is... > > I have 3 other processors for these things, as the hold a total of 4. I > decided to do my initial load with only 2 processors per box just to test > and get familiar with them (no need to unpack what I may not be able to use). > I've enabled SMP in the kernel config, and at boot time all goes well until > the 2nd processor is loaded. > > DDB shows it halt after loading: > > db> n > After 2 instructions (0 loads, 0 stores), > Stopped at atkbd_isa_intr+0x19: ret > db> n > After 248 instructions (0 loads, 0 stores), > Stopped at doreti_iret: iret > db> n > ....... > > And the kernel halts. Booting with a GENERIC kernel shows that the next > device to load is the pass1 device... this never happens with SMP enabled > and the last message seen before halting is "Waiting 15 seconds for SCSI > devices to settle". > > I've seen in the archives that alot of people have used these boxes in the > past successfully with an SMP kernel, but I can't seem to find anything > specific to the 6000R model. Currently, my kernel config is the GENERIC > kernel with SMP enabled. It made the most sense to me since is the first > time I've used one of these boxes, and the GENERIC kernel worked just fine. > > My SMP configs are: > > options SMP > options APIC_IO > options NCPU=2 > options NBUS=10 > options NAPIC=2 > options NINTR=39 > > Below are output from dmesg with the generic kernel and mptable output. > > Does anyone have any ideas? > > Thanks, > Denver > > ---------------------------------------------------------------- > > > Copyright (c) 1992-2000 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD 4.1-RELEASE #0: Fri Jul 28 14:30:31 GMT 2000 > jkh@ref4.freebsd.org:/usr/src/sys/compile/GENERIC > Timecounter "i8254" frequency 1193182 Hz > CPU: Unknown 80686 (701.62-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0x6a1 Stepping = 1 > Features=0x383fbff > real memory = 1073713152 (1048548K bytes) > avail memory = 1040621568 (1016232K bytes) > Pentium Pro MTRR support enabled > md0: Malloc disk > npx0: on motherboard > npx0: INT 16 interface > pcib2: on motherboard > pci2: on pcib2 > ahc0: port 0x4000-0x40ff mem 0xefbff000-0xefbfffff irq 10 at device 1.0 on pci2 > ahc0: aic7899 Wide Channel A, SCSI Id=7, 16/255 SCBs > ahc1: port 0x4100-0x41ff mem 0xefbfe000-0xefbfefff irq 10 at device 1.1 on pci2 > ahc1: aic7899 Wide Channel B, SCSI Id=7, 16/255 SCBs > pcib0: on motherboard > pci0: on pcib0 > fxp0: port 0x2200-0x223f mem 0xfea00000-0xfeafffff,0xfebff000-0xfebfffff irq 11 at device 1.0 on pci0 > fxp0: Ethernet address 00:06:29:8f:29:c2 > lnc0: port 0x2240-0x225f mem 0xfebfec00-0xfebfec1f irq 11 at device 5.0 on pci0 > lnc0: driver is using old-style compatability shims > pci0: at 6.0 > isab0: at device 15.0 on pci0 > isa0: on isab0 > atapci0: port 0x700-0x70f at device 15.1 on pci0 > ata0: at 0x1f0 irq 14 on atapci0 > ohci0: mem 0xfebfd000-0xfebfdfff irq 15 at device 15.2 on pci0 > usb0: OHCI version 1.0, legacy support > usb0: on ohci0 > usb0: USB revision 1.0 > uhub0: (unknown) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 > uhub0: 4 ports with 4 removable, self powered > pcib3: on motherboard > pci3: on pcib3 > pcib5: on motherboard > pci5: on pcib5 > fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 > fdc0: FIFO enabled, 8 bytes threshold > fd0: <1440-KB 3.5" drive> on fdc0 drive 0 > atkbdc0: at port 0x60,0x64 on isa0 > atkbd0: flags 0x1 irq 1 on atkbdc0 > kbd0 at atkbd0 > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 > sio0: type 16550A > sio1 at port 0x2f8-0x2ff irq 3 on isa0 > sio1: type 16550A > ppc0: at port 0x378-0x37f irq 7 on isa0 > ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode > ppi0: on ppbus0 > lpt0: on ppbus0 > lpt0: Interrupt-driven port > plip0: on ppbus0 > acd0: CDROM at ata0-master using PIO4 > Waiting 15 seconds for SCSI devices to settle > pass1 at ahc1 bus 0 target 8 lun 0 > pass1: Fixed Processor SCSI-2 device > pass1: 3.300MB/s transfers > da0 at ahc1 bus 0 target 2 lun 0 > da0: Fixed Direct Access SCSI-3 device > da0: 80.000MB/s transfers (40.000MHz, offset 31, 16bit), Tagged Queueing Enabled > da0: 17357MB (35548320 512 byte sectors: 255H 63S/T 2212C) > Mounting root from ufs:da0s1a > > =============================================================================== > > MPTable, version 2.0.15 > > ------------------------------------------------------------------------------- > > MP Floating Pointer Structure: > > location: EBDA > physical address: 0x0009e1d0 > signature: '_MP_' > length: 16 bytes > version: 1.4 > checksum: 0xd6 > mode: Virtual Wire > > ------------------------------------------------------------------------------- > > MP Config Table Header: > > physical address: 0x0009e1e0 > signature: 'PCMP' > base table length: 508 > version: 1.4 > checksum: 0x04 > OEM ID: 'UNISYS ' > Product ID: ' ' > OEM table pointer: 0x00000000 > OEM table size: 0 > entry count: 55 > local APIC address: 0xfee00000 > extended table length: 228 > extended table checksum: 158 > > ------------------------------------------------------------------------------- > > MP Config Base Table Entries: > > -- > Processors: APIC ID Version State Family Model Step Flags > 3 0x11 BSP, usable 6 10 1 0x0301 > 0 0x11 AP, usable 6 10 1 0x0301 > -- > Bus: Bus ID Type > 0 PCI > 1 PCI > 2 PCI > 3 PCI > 4 PCI > 5 PCI > 6 PCI > 7 PCI > 8 PCI > 9 ISA > -- > I/O APICs: APIC ID Version State Address > 14 0x11 usable 0xfec00000 > 13 0x11 usable 0xfec01000 > -- > I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# > INT conforms conforms 9 1 14 1 > INT conforms conforms 9 0 14 2 > INT conforms conforms 9 3 14 3 > INT conforms conforms 9 4 14 4 > INT active-lo level 9 5 14 5 > INT conforms conforms 9 6 14 6 > INT conforms conforms 9 7 14 7 > INT conforms conforms 9 8 14 8 > INT conforms conforms 9 12 14 12 > INT conforms conforms 9 13 14 13 > INT conforms conforms 9 14 14 14 > INT conforms conforms 0 1:A 13 4 > INT conforms conforms 0 5:A 13 0 > INT conforms conforms 0 15:A 13 3 > INT conforms conforms 2 1:A 13 1 > INT conforms conforms 2 1:B 13 2 > INT conforms conforms 0 1:B 14 15 > INT conforms conforms 0 1:C 14 9 > INT conforms conforms 0 1:D 14 9 > INT conforms conforms 5 2:A 13 5 > INT conforms conforms 5 2:B 14 9 > INT conforms conforms 5 2:C 14 9 > INT conforms conforms 5 2:D 14 9 > INT conforms conforms 5 3:A 13 6 > INT conforms conforms 5 3:B 14 10 > INT conforms conforms 5 3:C 14 9 > INT conforms conforms 5 3:D 14 9 > INT conforms conforms 5 4:A 13 7 > INT conforms conforms 5 4:B 14 11 > INT conforms conforms 5 4:C 14 9 > INT conforms conforms 5 4:D 14 9 > INT conforms conforms 2 5:A 13 8 > INT conforms conforms 2 5:B 14 15 > INT conforms conforms 2 5:C 14 9 > INT conforms conforms 2 5:D 14 9 > INT conforms conforms 2 6:A 13 9 > INT conforms conforms 2 6:B 14 9 > INT conforms conforms 2 6:C 14 9 > INT conforms conforms 2 6:D 14 9 > -- > Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN# > NMI conforms conforms 9 0 255 1 > ExtINT conforms conforms 9 0 255 0 > > ------------------------------------------------------------------------------- > > MP Config Extended Table Entries: > > -- > > bus ID: 0 address type: memory address > address base: 0xa0000 > address range: 0x20000 > -- > > bus ID: 0 address type: memory address > address base: 0xc0000 > address range: 0x20000 > -- > > bus ID: 0 address type: prefetch address > address base: 0xefc00000 > address range: 0x8a00000 > -- > > bus ID: 0 address type: memory address > address base: 0xf8600000 > address range: 0x7a00000 > -- > > bus ID: 2 address type: prefetch address > address base: 0xed000000 > address range: 0x800000 > -- > > bus ID: 2 address type: memory address > address base: 0xed800000 > address range: 0x2400000 > -- > > bus ID: 5 address type: memory address > address base: 0xeb800000 > address range: 0xc00000 > -- > > bus ID: 5 address type: prefetch address > address base: 0xec400000 > address range: 0xc00000 > -- > > bus ID: 0 address type: I/O address > address base: 0x0 > address range: 0x4000 > -- > > bus ID: 2 address type: I/O address > address base: 0x4000 > address range: 0x3000 > -- > > bus ID: 5 address type: I/O address > address base: 0x7000 > address range: 0x9000 > -- > > bus ID: 9 bus info: 0x01 parent bus ID: 0 > ------------------------------------------------------------------------------- > > # SMP kernel config file options: > > > # Required: > options SMP # Symmetric MultiProcessor Kernel > options APIC_IO # Symmetric (APIC) I/O > > # Optional (built-in defaults will work in most cases): > #options NCPU=2 # number of CPUs > #options NBUS=10 # number of busses > #options NAPIC=2 # number of IO APICs > #options NINTR=39 # number of INTs > > =============================================================================== > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Sep 26 1:32:56 2000 Delivered-To: freebsd-smp@freebsd.org Received: from anchor-post-33.mail.demon.net (anchor-post-33.mail.demon.net [194.217.242.91]) by hub.freebsd.org (Postfix) with ESMTP id 7BE6C37B424; Tue, 26 Sep 2000 01:32:48 -0700 (PDT) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by anchor-post-33.mail.demon.net with esmtp (Exim 2.12 #1) id 13dqAQ-0003bt-0X; Tue, 26 Sep 2000 09:32:47 +0100 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id JAA74959; Tue, 26 Sep 2000 09:36:51 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Tue, 26 Sep 2000 09:32:32 +0100 (BST) From: Doug Rabson To: John Baldwin Cc: smp@freebsd.org, cp@bsdi.com, alpha@freebsd.org Subject: Re: Status update In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sat, 23 Sep 2000, John Baldwin wrote: > Ok, the alpha seems to be rather stable now without the need for obscene hacks > to the mutex code to dink with mtx_saveipl. To summarize, here are the changes > thus far: > > - The interrupt state of the sched_lock is now saved in a process's PCB during > cpu_switch(). This way, code before and after a call to either mi_switch() > or cpu_switch() is guaranteed to be run at the same interrupt state. Without > this I was having problems on the alpha where the idle loop was running at > ALPHA_PSL_IPL_SOFT (1) and as a result init's child process was never ran, > among other things. > > This last change is something I'd like some feedback on. I've checked > the BSD/OS x86 code, and it onyl saves the recursion count of the > sched_lock in the pcb. However, after the problems with the alpha and > some discussion with Peter Wemm on IRC, I decided that we should be > doing this. However, I'm not completely certain, and any thoughts > that anyone has would be appreciated. I think this is probably unnecessary. After implementing swi threads, the only place in the kernel where ipl will be non-zero when sched_lock is taken should be in the interrupt code and I don't think a context switch is possible there until the AST after returning. > > There are also a few more weirdism's on the alpha. In a few places in > sys/kern, we call spl0() instead of splx(). I've added some debugging code to > do a printf() if we aren't actually at IPL_0 (what spl0 used to do) after the > mtx_exit(). It does trigger in several cases during /etc/rc at least, but the > machine seems to be running stable regardless (I'll be running a buildworld -j > 8 tonight to stress test it). My question is: is it ok for the code to run > with some interrupts disabled or do we need to replace the calls to spl0() > with enable_intr()? I'm testing this now and I'm seeing a flood of diagnostic messages like: ../../kern/kern_fork.c:537:fork1() spl0 needs fixing I think these are all due to the fact that sched_lock is held at that point. The preceding mtx_exit() just decrements the recursion count. I haven't worked out exactly why sched_lock is held here because it shouldn't be. Possibly its because we don't clear pcb_schednest in cpu_fork(). -- Doug Rabson Mail: dfr@nlsystems.com Phone: +44 20 8348 6160 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Sep 26 1:49:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from finch-post-11.mail.demon.net (finch-post-11.mail.demon.net [194.217.242.39]) by hub.freebsd.org (Postfix) with ESMTP id A34AC37B42C; Tue, 26 Sep 2000 01:49:24 -0700 (PDT) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by finch-post-11.mail.demon.net with esmtp (Exim 2.12 #1) id 13dqQS-000B5T-0B; Tue, 26 Sep 2000 08:49:23 +0000 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id JAA75095; Tue, 26 Sep 2000 09:53:49 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Tue, 26 Sep 2000 09:49:30 +0100 (BST) From: Doug Rabson To: John Baldwin Cc: smp@freebsd.org, cp@bsdi.com, alpha@freebsd.org Subject: Re: Status update In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 26 Sep 2000, Doug Rabson wrote: > On Sat, 23 Sep 2000, John Baldwin wrote: > > > > There are also a few more weirdism's on the alpha. In a few places in > > sys/kern, we call spl0() instead of splx(). I've added some debugging code to > > do a printf() if we aren't actually at IPL_0 (what spl0 used to do) after the > > mtx_exit(). It does trigger in several cases during /etc/rc at least, but the > > machine seems to be running stable regardless (I'll be running a buildworld -j > > 8 tonight to stress test it). My question is: is it ok for the code to run > > with some interrupts disabled or do we need to replace the calls to spl0() > > with enable_intr()? > > I'm testing this now and I'm seeing a flood of diagnostic messages like: > > ../../kern/kern_fork.c:537:fork1() spl0 needs fixing > > I think these are all due to the fact that sched_lock is held at that > point. The preceding mtx_exit() just decrements the recursion count. > I haven't worked out exactly why sched_lock is held here because it > shouldn't be. Possibly its because we don't clear pcb_schednest in > cpu_fork(). As I suspected, clearing pcb_schednest makes this lot go away. Try this version of vm_machdep.c: Index: vm_machdep.c =================================================================== RCS file: /home/ncvs/src/sys/alpha/alpha/vm_machdep.c,v retrieving revision 1.33 diff -u -r1.33 vm_machdep.c --- vm_machdep.c 2000/09/07 01:32:39 1.33 +++ vm_machdep.c 2000/09/26 08:38:55 @@ -210,6 +210,13 @@ up->u_pcb.pcb_context[2] = (u_long) p2; /* s2: a0 */ up->u_pcb.pcb_context[7] = (u_int64_t)switch_trampoline; /* ra: assembly magic */ + + /* + * Clear the saved recursion count for sched_lock + * since the child needs only one count which is + * released in switch_trampoline. + */ + up->u_pcb.pcb_schednest = 0; } } -- Doug Rabson Mail: dfr@nlsystems.com Phone: +44 20 8348 6160 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Sep 27 22:28:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 9E0A237B424 for ; Wed, 27 Sep 2000 22:28:40 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id XAA14206 for ; Wed, 27 Sep 2000 23:28:38 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id XAA66502 for ; Wed, 27 Sep 2000 23:28:38 -0600 (MDT) Message-Id: <200009280528.XAA66502@harmony.village.org> To: smp@freebsd.org Subject: Kernel threads Date: Wed, 27 Sep 2000 23:28:38 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org It appears that kernel threads are not running any more. At least my newcard thread isn't running after the smpng changes. Any ideas on where I can look to see what might be wrong? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Sep 28 0:49:41 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id C1B7737B422 for ; Thu, 28 Sep 2000 00:49:33 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id BAA14597 for ; Thu, 28 Sep 2000 01:49:31 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id BAA67360 for ; Thu, 28 Sep 2000 01:49:31 -0600 (MDT) Message-Id: <200009280749.BAA67360@harmony.village.org> To: smp@freebsd.org Subject: Releasing interrupts? Date: Thu, 28 Sep 2000 01:49:31 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I've also noticed that with OLDCARD when I eject a card and insert another card that the new card doesn't get interrupts. This is with drivers that are known goon and worked before. I've not tried to debug this, but since the interrupt threads went in with the latest SMPng I thought I'd post here. If this is the wrong place, lemme know. I'll debug it later today. Dang. I had hoped to get NEWCARD probe/attaching cards tonight. Such is the price of current. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Sep 28 13:11:49 2000 Delivered-To: freebsd-smp@freebsd.org Received: from troutmask.apl.washington.edu (troutmask.apl.washington.edu [128.208.78.105]) by hub.freebsd.org (Postfix) with ESMTP id 03EAA37B422 for ; Thu, 28 Sep 2000 13:11:44 -0700 (PDT) Received: (from sgk@localhost) by troutmask.apl.washington.edu (8.11.0/8.11.0) id e8SKHPg00809 for freebsd-smp@freebsd.org; Thu, 28 Sep 2000 13:17:25 -0700 (PDT) (envelope-from sgk) From: Steve Kargl Message-Id: <200009282017.e8SKHPg00809@troutmask.apl.washington.edu> Subject: 2 panics with MP kernels To: freebsd-smp@freebsd.org Date: Thu, 28 Sep 2000 13:17:25 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL61 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In attempting to boot a MP kernel from today's sources (28 Sep 00, 9:30pst), I get 2 panics. Yes, I know -current will experience periods of instabilities, so this report is a FYI with what info I could get. Panic 1 is related to havig "options USER_LDT" in the kernel config file. Info I have so far is (transcribed by hand): Fatal trap 12: page fault while in kernel mode cpuid = 0; lapic.id = 0100000 fault virtual address = 0x17c fault code = supervisor read, page not present instruction pointer = 0x8:0xc0258196 stack pointer = 0x10:0xcb355f00 frame pointer = 0x10:0xcb355f30 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL0, pres 1, def32 1, gran 1 processor flags = resume, IOPL=0 current process = 12 (random) kernel: type trap 12, code = 0 CPU0 stopping CPUs: 0x00000002... stopped Stopped at sw1b+0xc: movl 0x17c(%ecx),%edx db> trace sw1b msleep random_kthread fork_trampoline I have not been able to get a crash dump. If I remove "options USER_LDT" from the kernel config file, I get the following panic (wrapped to fit < 80columns): panic: mutex_enter: MTX_DEF on MTX_SPIN mutex random harvest\ @/usr/src/sys/modules/randomdev/../../dev/randomdev/yarrow.c:515 cpuid = 1; lapic.id = 00000000 Debugger("panic') CPU1 stopping CPUs: 0x00000001 stopped Stopped at Debugger+0x38: movb $0,in_Debugger.684 db>trace Debugger(c0277da1) panic witness_enter _mtx_enter random_harvest_internal write_random random_write ufs_vnoperatespec vn_write dofilewrite write syscall2 Xint0x80_syscall Again, I haven't been able to convince the system to produce a crash dump. -- Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Sep 28 17:18:49 2000 Delivered-To: freebsd-smp@freebsd.org Received: from midten.fast.no (midten.fast.no [213.188.8.11]) by hub.freebsd.org (Postfix) with ESMTP id 1C4ED37B440 for ; Thu, 28 Sep 2000 17:17:04 -0700 (PDT) Received: from fast.no (IDENT:tegge@midten.fast.no [213.188.8.11]) by midten.fast.no (8.9.3/8.9.3) with ESMTP id CAA45617; Fri, 29 Sep 2000 02:16:58 +0200 (CEST) Message-Id: <200009290016.CAA45617@midten.fast.no> To: sgk@troutmask.apl.washington.edu Cc: freebsd-smp@FreeBSD.ORG Subject: Re: 2 panics with MP kernels From: Tor.Egge@fast.no In-Reply-To: Your message of "Thu, 28 Sep 2000 13:17:25 -0700 (PDT)" References: <200009282017.e8SKHPg00809@troutmask.apl.washington.edu> X-Mailer: Mew version 1.70 on Emacs 19.34.1 Mime-Version: 1.0 Content-Type: Multipart/Mixed; boundary="--Next_Part(Fri_Sep_29_02:16:07_2000)--" Content-Transfer-Encoding: 7bit Date: Fri, 29 Sep 2000 02:16:58 +0200 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org ----Next_Part(Fri_Sep_29_02:16:07_2000)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit > Again, I haven't been able to convince the system to > produce a crash dump. Currently, recursive panics on SMP machines causes infinite recursion due to s_lock calling panic if the lock is already held by the same CPU. The "panic" command on the ddb prompt only results in a "Fatal double fault". The enclosed patch might help. - Tor Egge ----Next_Part(Fri_Sep_29_02:16:07_2000)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Index: kern_shutdown.c =================================================================== RCS file: /home/ncvs/src/sys/kern/kern_shutdown.c,v retrieving revision 1.83 diff -u -r1.83 kern_shutdown.c --- kern_shutdown.c 2000/09/17 12:20:49 1.83 +++ kern_shutdown.c 2000/09/25 12:19:16 @@ -67,6 +67,7 @@ #include #include #include /* smp_active, cpuid */ +#include #include @@ -541,6 +542,10 @@ #ifdef SMP /* Only 1 CPU can panic at a time */ +#ifdef __i386__ + /* Avoid fatal double fault */ + if (panic_lock.lock_data != GLOBALDATA->gd_cpu_lockid) +#endif s_lock(&panic_lock); #endif ----Next_Part(Fri_Sep_29_02:16:07_2000)---- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Sep 28 19:24:57 2000 Delivered-To: freebsd-smp@freebsd.org Received: from volatile.chemicals.tacorp.com (ci391991-a.grnvle1.sc.home.com [24.9.31.75]) by hub.freebsd.org (Postfix) with ESMTP id 9A48837B446; Thu, 28 Sep 2000 19:20:53 -0700 (PDT) Received: (from morganw@localhost) by volatile.chemicals.tacorp.com (8.11.0/8.11.0) id e8T2Kq171337; Thu, 28 Sep 2000 22:20:52 -0400 (EDT)?g (envelope-from morganw)œ Date: Thu, 28 Sep 2000 22:20:52 -0400 (EDT) From: Wesley Morgan To: current@freebsd.org Cc: smp@freebsd.org Subject: panic in "kernel configuration menu" Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org When the kernel configuration menu comes up with the three possible selections, pressing ctrl-alt-del ends up with this message: panic: spin lock (null) held by 0x0 for > 5 seconds sounds like one that should be an easy fix -- _ __ ___ ____ ___ ___ ___ Wesley N Morgan _ __ ___ | _ ) __| \ wesleymorgan@home.com _ __ | _ \._ \ |) | FreeBSD: The Power To Serve _ |___/___/___/ 6bone: 3ffe:1ce3:7::b4ff:fe53:c297 Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Sep 28 20:24:44 2000 Delivered-To: freebsd-smp@freebsd.org Received: from wantadilla.lemis.com (wantadilla.lemis.com [192.109.197.80]) by hub.freebsd.org (Postfix) with ESMTP id 6B8B737B424; Thu, 28 Sep 2000 20:24:15 -0700 (PDT) Received: (from grog@localhost) by wantadilla.lemis.com (8.11.0/8.9.3) id e8T3LWQ04932; Fri, 29 Sep 2000 12:51:32 +0930 (CST) (envelope-from grog) Date: Fri, 29 Sep 2000 12:51:32 +0930 From: Greg Lehey To: Wesley Morgan Cc: current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: panic in "kernel configuration menu" Message-ID: <20000929125132.H4293@wantadilla.lemis.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from morganw@chemicals.tacorp.com on Thu, Sep 28, 2000 at 10:20:52PM -0400 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thursday, 28 September 2000 at 22:20:52 -0400, Wesley Morgan wrote: > When the kernel configuration menu comes up with the three possible > selections, pressing ctrl-alt-del ends up with this message: > > panic: spin lock (null) held by 0x0 for > 5 seconds > > sounds like one that should be an easy fix Don't count on it. At the moment, it probably fits into the category "well don't do that then". Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Sep 30 19:13:17 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.rdc1.va.home.com (ha1.rdc1.va.home.com [24.2.32.66]) by hub.freebsd.org (Postfix) with ESMTP id 7CB2B37B691 for ; Sat, 30 Sep 2000 19:13:09 -0700 (PDT) Received: from laptop.baldwin.cx ([24.6.244.187]) by mail.rdc1.va.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20001001021308.NTMC26082.mail.rdc1.va.home.com@laptop.baldwin.cx>; Sat, 30 Sep 2000 19:13:08 -0700 Content-Length: 2519 Message-ID: X-Mailer: XFMail 1.4.0 on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <200009280749.BAA67360@harmony.village.org> Date: Sat, 30 Sep 2000 19:13:09 -0700 (PDT) From: John Baldwin To: Warner Losh Subject: RE: Releasing interrupts? Cc: smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 28-Sep-00 Warner Losh wrote: > I've also noticed that with OLDCARD when I eject a card and insert > another card that the new card doesn't get interrupts. This is with > drivers that are known goon and worked before. Grr. I'd test it on my laptop, but pccard isnt' attaching to my pcic_pci controller. I've tried the following patch but no go: Index: pccard/pccard_nbk.c =================================================================== RCS file: /usr/cvs/src/sys/pccard/pccard_nbk.c,v retrieving revision 1.23 diff -u -r1.23 pccard_nbk.c --- pccard/pccard_nbk.c 2000/09/28 07:22:30 1.23 +++ pccard/pccard_nbk.c 2000/09/28 16:19:48 @@ -390,4 +390,5 @@ DRIVER_MODULE(pccard, pcic, pccard_driver, pccard_devclass, 0, 0); DRIVER_MODULE(pccard, pc98pcic, pccard_driver, pccard_devclass, 0, 0); DRIVER_MODULE(pccard, cbb, pccard_driver, pccard_devclass, 0, 0); +DRIVER_MODULE(pccard, pcic_pci, pccard_driver, pccard_devclass, 0, 0); MODULE_VERSION(pccard, 1); cvs diff: Diffing dev/pccard Index: dev/pccard/pccard.c =================================================================== RCS file: /usr/cvs/src/sys/dev/pccard/pccard.c,v retrieving revision 1.18 diff -u -r1.18 pccard.c --- dev/pccard/pccard.c 2000/09/22 01:15:26 1.18 +++ dev/pccard/pccard.c 2000/09/28 03:15:01 @@ -789,5 +789,6 @@ DRIVER_MODULE(pccard, pc98pcic, pccard_driver, pccard_devclass, 0, 0); DRIVER_MODULE(pccard, pccbb, pccard_driver, pccard_devclass, 0, 0); DRIVER_MODULE(pccard, tcic, pccard_driver, pccard_devclass, 0, 0); +DRIVER_MODULE(pccard, pcic_pci, pccard_driver, pccard_devclass, 0, 0); MODULE_VERSION(pccard, 1); MODULE_DEPEND(pccard, pcic, 1, 1, 1); dmesg output: pcic-pci0: at device 4.0 on pci0 pcic-pci0: TI12XX PCI Config Reg: [ring enable][speaker enable][pwr save][FUNC pci int + CSC serial isa irq] pcic-pci1: at device 4.1 on pci0 pcic-pci1: TI12XX PCI Config Reg: [ring enable][speaker enable][pwr save][FUNC pci int + CSC serial isa irq] > I've not tried to debug this, but since the interrupt threads went in > with the latest SMPng I thought I'd post here. If this is the wrong > place, lemme know. > > I'll debug it later today. Dang. I had hoped to get NEWCARD > probe/attaching cards tonight. Such is the price of current. > > Warner -- John Baldwin -- http://www.FreeBSD.org/~jhb/ PGP Key: http://www.baldwin.cx/~john/pgpkey.asc "Power Users Use the Power to Serve!" - http://www.FreeBSD.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Sep 30 20:31:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id F0F7E37B66D; Sat, 30 Sep 2000 20:31:22 -0700 (PDT) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id VAA28160; Sat, 30 Sep 2000 21:31:21 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id VAA14982; Sat, 30 Sep 2000 21:31:21 -0600 (MDT) Message-Id: <200010010331.VAA14982@harmony.village.org> To: John Baldwin Subject: Re: Releasing interrupts? Cc: smp@FreeBSD.org In-reply-to: Your message of "Sat, 30 Sep 2000 19:13:09 PDT." References: Date: Sat, 30 Sep 2000 21:31:21 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message John Baldwin writes: : Grr. I'd test it on my laptop, but pccard isnt' attaching to my : pcic_pci controller. I've tried the following patch but no go: The reason is that the pcic_p.c isn't putting the cardbus bridge into legacy mode correctly. I'd bet a case of beer that the patch you included won't work. : pcic-pci0: at device 4.0 on pci0 These are always a pita. what kind of laptop do you have? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message