From owner-freebsd-smp Sun Dec 1 00:21:08 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id AAA06161 for smp-outgoing; Sun, 1 Dec 1996 00:21:08 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id AAA06156 for ; Sun, 1 Dec 1996 00:21:06 -0800 (PST) Received: from critter.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0vU78X-0003vpC; Sun, 1 Dec 96 00:20 PST Received: from critter.tfs.com (localhost.phk.dk [127.0.0.1]) by critter.tfs.com (8.8.2/8.8.2) with ESMTP id JAA00277; Sun, 1 Dec 1996 09:21:42 +0100 (MET) To: Gunther Hipper cc: freebsd-smp@freebsd.org Subject: Re: New smp kernel / Panic(cpu#0): Nobody wants to mount my root for me In-reply-to: Your message of "Sun, 01 Dec 1996 01:03:24 +0100." <199612010003.BAA04830@donau.informatik.uni-rostock.de> Date: Sun, 01 Dec 1996 09:21:42 +0100 Message-ID: <275.849428502@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk In message <199612010003.BAA04830@donau.informatik.uni-rostock.de>, Gunther Hip per writes: >Hi ! > >Great news - since 4 hours I've no girlfriend, but maybe an _SMP_kernel_ ! >That's better than nothing ! Still a pretty bad swap I'd say... -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Sun Dec 1 10:29:06 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA24601 for smp-outgoing; Sun, 1 Dec 1996 10:29:06 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id KAA24595 for ; Sun, 1 Dec 1996 10:29:03 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id TAA11629; Sun, 1 Dec 1996 19:28:55 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id TAA05410; Sun, 1 Dec 1996 19:28:54 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id TAA08278; Sun, 1 Dec 1996 19:28:53 +0100 (MET) Message-Id: <199612011828.TAA08278@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Steve Passe cc: freebsd-smp@freebsd.org Subject: Re: SMP-current status In-reply-to: Your message of "Sat, 30 Nov 1996 19:38:15 MET." <199612010238.TAA10184@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Sun, 01 Dec 1996 19:28:52 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >Status: > > 1: I think that I broke something when I merged the global used > for INTerrupt masking by the 8259s (imen) and the global > used by the APIC_IO code (IOApicMask). But I am clueless as to what. > I just rebuilt a non APIC_IO kernel that works well. But I have > no IDE disk so I have no way of testing that aspect of it. > > 2: Peter commited a fix for this today but we haven't heard back from > any >2 CPU testers yet. Just tried on our HP Netserver 5/133 LS4. However, unlike Erich, which seems to have gotten his all his processors started, I still can't get more than two. First I had the: BIOS basemem (633K) != RTC basemem (640K), setting to BIOS value unknown bus type: 'XPRESS' Where I simply commented out the panic. But then all stops with: Warning: current SMP kernel only tested with 2 CPUs. Please report the results to: to continue... Application Processor #3 failed! panic (cpu#0): Just for fun I removed the panic(), put in a couple more debugs and ended up with: Warning: current SMP kernel only tested with 2 CPUs. Please report the results to: to continue... Application Processor #2 starting! Application Processor #3 starting! Application Processor #3 failed! Application Processor #4 starting! Application Processor #4 failed! Sizing memory.. init386 done CR0 = 80000011 Copyright (c) 1992-1996 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.0-SMP #0: Sun Dec 1 18:25:33 MET 1996 terjem@quattro:/usr/src/sys/compile/netserver FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010 cpu1 (AP): apic id: 2, version: 0x00030010 cpu2 (AP): apic id: 3, version: 0x00030010 cpu3 (AP): apic id: 4, version: 0x00030010 io0 (APIC): apic id: 14, version: 0x000f0011 It then continues to boot, puts #2 into the scheduler and comes up just fine. I haven't any other OS on that machine right now, but I moved the disk to another identical machine (btw. it had stepping 11 on CPU 3 and 4) and got the same results, so I don't think it's the hardware thats faulty either. Tried to compile it with options APIC_IO for the first time as well: BIOS basemem (633K) != RTC basemem (640K), setting to BIOS value unknown bus type: 'XPRESS' Warning: current SMP kernel only tested with 2 CPUs. Please report the results to: to continue... Application Processor #2 starting! Application Processor #3 starting! Application Processor #3 failed! Application Processor #4 starting! Application Processor #4 failed! Sizing memory.. init386 done CR0 = 80000011 Copyright (c) 1992-1996 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.0-SMP #0: Sun Dec 1 18:25:33 MET 1996 terjem@quattro:/usr/src/sys/compile/netserver FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010 cpu1 (AP): apic id: 2, version: 0x00030010 cpu2 (AP): apic id: 3, version: 0x00030010 cpu3 (AP): apic id: 4, version: 0x00030010 io0 (APIC): apic id: 14, version: 0x000f0011 Calibrating clock(s) relative to mc146818A clock ... i8254 clock: 1193177 Hz CPU: Pentium (586-class CPU) Origin = "GenuineIntel" Id = 0x52b Stepping=11 Features=0x3bf real memory = 134217728 (131072K bytes) avail memory = 127418368 (124432K bytes) eisa0: Probing for devices on the EISA bus Probing for devices on PCI bus 0: chip0 rev 2 on pci0:0 chip1 rev 5 on pci0:14:0 pci0:15:0: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] pci0:15:1: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] pci0:15:2: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] pci0:15:3: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] pci0:15:4: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] pci0:15:5: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] pci0:15:6: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] pci0:15:7: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no driver assigned] Probing for devices on PCI bus 1: chip2 rev 2 on pci1:0 vx0 <3COM 3C595 Fast Etherlink XL PCI> rev 0 int a irq 11 on pci1:12 mii[*mii*]: disable 'auto select' with DOS util! address 00:60:97:12:60:e8 ahc0 rev 3 int a irq 7 on pci1:13 ahc0: Using left over BIOS settings ahc0: aic7870 Wide Channel, SCSI Id=7, 16 SCBs ahc0 waiting for scsi devices to settle Sending SDTR!! (ahc0:0:0): "HP 2.13 GB 1st ### 1221" type 0 fixed SCSI 2 sd0(ahc0:0:0): Direct-Access 2033MB (4165272 512 byte sectors) (ahc0:5:0): "TOSHIBA CD-ROM XM-5301TA 1895" type 5 removable SCSI 2 cd0(ahc0:5:0): CD-ROM cd0(ahc0:5:0): NOT READY asc:3a,0 Medium not present can't get the size ahc1 rev 3 int a irq 10 on pci1:14 ahc1: Using left over BIOS settings ahc1: aic7870 Wide Channel, SCSI Id=6, 16 SCBs ahc1 waiting for scsi devices to settle Sending SDTR!! (ahc1:4:0): "HP 2.13 GB 1st ### 1221" type 0 fixed SCSI 2 sd1(ahc1:4:0): Direct-Access 2033MB (4165272 512 byte sectors) Probing for devices on the ISA bus: sc0 at 0x60-0x6f irq 1 on motherboard sc0: VGA color <16 virtual consoles, flags=0x0> sio0 at 0x3f8-0x3ff irq 4 on isa sio0: type 16550A sio1 not found at 0x2f8 lpt0 not found at 0xffffffff mse0 not found at 0x23c fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: NEC 72065B fd0: 1.44MB 3.5in wdc0 not found at 0x1f0 npx0 on motherboard npx0: INT 16 interface stray irq 13 changing root device to sd0a Enabled INTs: 1, 2, 4, 6, 7, 8, 10, 11, 13, imen: 0x00ffd229 Rebooting... It then just hang at Enabled. "Rebooting..." came after ctrl-alt-delete Sorry, would have liked to do some better debugging than this, but it's closing in on my most import exam this year, and I also have an IP across cable TV test project to finish at work, so time really dosn't allow to get the needed knowledge of FreeBSD SMP work in at least a couple of weeks. I'll try to get time to do any testing you need me to do to get things working though. Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Sun Dec 1 11:28:02 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA26862 for smp-outgoing; Sun, 1 Dec 1996 11:28:02 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA26849 for ; Sun, 1 Dec 1996 11:27:59 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id MAA14928; Sun, 1 Dec 1996 12:27:47 -0700 Message-Id: <199612011927.MAA14928@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terje Normann Marthinussen cc: freebsd-smp@freebsd.org Subject: Re: SMP-current status In-reply-to: Your message of "Sun, 01 Dec 1996 19:28:52 +0100." <199612011828.TAA08278@slibo.cc.uit.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 01 Dec 1996 12:27:46 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >unknown bus type: 'XPRESS' > >Where I simply commented out the panic. I need to look this over to see what might be damaged by continuing at this point. --- >But then all stops with: >Warning: current SMP kernel only tested with 2 CPUs. >Please report the results to: > to continue... >Application Processor #3 failed! >panic (cpu#0): > > >Just for fun I removed the panic(), put in a couple more debugs and ended >up with: >Warning: current SMP kernel only tested with 2 CPUs. >Please report the results to: > to continue... >Application Processor #2 starting! >Application Processor #3 starting! >Application Processor #3 failed! >Application Processor #4 starting! >Application Processor #4 failed! this is a fundimental problem. there may or may not be additional failures and/or clues beyound this point, but till we fix the above 2 ERRORs it probably would be poor use of time to chase them, as they most likely would be bogus (ie go away when the above are fixed). I'll go look at the XXPRESS bus thing first... -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Sun Dec 1 12:08:45 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA28357 for smp-outgoing; Sun, 1 Dec 1996 12:08:45 -0800 (PST) Received: (from fsmp@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA28350 for freebsd-smp; Sun, 1 Dec 1996 12:08:43 -0800 (PST) Date: Sun, 1 Dec 1996 12:08:43 -0800 (PST) From: Steve Passe Message-Id: <199612012008.MAA28350@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mp_machdep.c sys/i386/include mpapic.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk fsmp 96/12/01 12:08:43 Modified: i386/i386 mp_machdep.c i386/include mpapic.h Log: added XPRESS bus to recognized types. bumped NBUS to 4 for "out of box" convenience. Revision Changes Path 1.24 +2 -2 sys/i386/i386/mp_machdep.c 1.7 +2 -7 sys/i386/include/mpapic.h From owner-freebsd-smp Sun Dec 1 12:36:50 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA29534 for smp-outgoing; Sun, 1 Dec 1996 12:36:50 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA29527 for ; Sun, 1 Dec 1996 12:36:47 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id VAA12539 for ; Sun, 1 Dec 1996 21:36:44 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id VAA05730 for ; Sun, 1 Dec 1996 21:36:43 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id VAA10981 for ; Sun, 1 Dec 1996 21:36:42 +0100 (MET) Message-Id: <199612012036.VAA10981@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: freebsd-smp@freebsd.org Subject: Re: SMP-current status In-reply-to: Your message of "Sun, 01 Dec 1996 19:28:52 MET." <199612011828.TAA08278@slibo.cc.uit.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 01 Dec 1996 21:36:41 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >Tried to compile it with options APIC_IO for the first time as well: >BIOS basemem (633K) != RTC basemem (640K), setting to BIOS value >unknown bus type: 'XPRESS' > >Warning: current SMP kernel only tested with 2 CPUs. >Please report the results to: > to continue... >Application Processor #2 starting! >Application Processor #3 starting! >Application Processor #3 failed! >Application Processor #4 starting! >Application Processor #4 failed! >Sizing memory.. >init386 done CR0 = 80000011 >Copyright (c) 1992-1996 FreeBSD Inc. >Copyright (c) 1982, 1986, 1989, 1991, 1993 > The Regents of the University of California. All rights reserved. > >FreeBSD 3.0-SMP #0: Sun Dec 1 18:25:33 MET 1996 > terjem@quattro:/usr/src/sys/compile/netserver >FreeBSD/SMP: Multiprocessor motherboard > cpu0 (BSP): apic id: 0, version: 0x00030010 > cpu1 (AP): apic id: 2, version: 0x00030010 > cpu2 (AP): apic id: 3, version: 0x00030010 > cpu3 (AP): apic id: 4, version: 0x00030010 > io0 (APIC): apic id: 14, version: 0x000f0011 >Calibrating clock(s) relative to mc146818A clock ... i8254 clock: 1193177 Hz >CPU: Pentium (586-class CPU) > Origin = "GenuineIntel" Id = 0x52b Stepping=11 > Features=0x3bf >real memory = 134217728 (131072K bytes) >avail memory = 127418368 (124432K bytes) >eisa0: >Probing for devices on the EISA bus >Probing for devices on PCI bus 0: >chip0 rev 2 on pci0:0 >chip1 rev 5 on pci0:14:0 >pci0:15:0: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >pci0:15:1: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >pci0:15:2: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >pci0:15:3: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >pci0:15:4: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >pci0:15:5: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >pci0:15:6: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >pci0:15:7: Intel Corporation, device=0x0008, class=0xff, subclass=0x00 [no >driver assigned] >Probing for devices on PCI bus 1: >chip2 rev 2 on pci1:0 >vx0 <3COM 3C595 Fast Etherlink XL PCI> rev 0 int a irq 11 on pci1:12 >mii[*mii*]: disable 'auto select' with DOS util! address 00:60:97:12:60:e8 >ahc0 rev 3 int a irq 7 on pci1:13 >ahc0: Using left over BIOS settings >ahc0: aic7870 Wide Channel, SCSI Id=7, 16 SCBs >ahc0 waiting for scsi devices to settle >Sending SDTR!! >(ahc0:0:0): "HP 2.13 GB 1st ### 1221" type 0 fixed SCSI 2 >sd0(ahc0:0:0): Direct-Access 2033MB (4165272 512 byte sectors) >(ahc0:5:0): "TOSHIBA CD-ROM XM-5301TA 1895" type 5 removable SCSI 2 >cd0(ahc0:5:0): CD-ROM >cd0(ahc0:5:0): NOT READY asc:3a,0 Medium not present >can't get the size >ahc1 rev 3 int a irq 10 on pci1:14 >ahc1: Using left over BIOS settings >ahc1: aic7870 Wide Channel, SCSI Id=6, 16 SCBs >ahc1 waiting for scsi devices to settle >Sending SDTR!! >(ahc1:4:0): "HP 2.13 GB 1st ### 1221" type 0 fixed SCSI 2 >sd1(ahc1:4:0): Direct-Access 2033MB (4165272 512 byte sectors) >Probing for devices on the ISA bus: >sc0 at 0x60-0x6f irq 1 on motherboard >sc0: VGA color <16 virtual consoles, flags=0x0> >sio0 at 0x3f8-0x3ff irq 4 on isa >sio0: type 16550A >sio1 not found at 0x2f8 >lpt0 not found at 0xffffffff >mse0 not found at 0x23c >fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa >fdc0: NEC 72065B >fd0: 1.44MB 3.5in >wdc0 not found at 0x1f0 >npx0 on motherboard >npx0: INT 16 interface >stray irq 13 >changing root device to sd0a >Enabled INTs: 1, 2, 4, 6, 7, 8, 10, 11, 13, imen: 0x00ffd229 >Rebooting... > > >It then just hang at Enabled. >"Rebooting..." came after ctrl-alt-delete This might be of a bit more help... I let it stay there for a couple of minutes the first time, but it seems like that wasn't enough, came to reboot again a bit later. Thought I had another kernel in /, went out of the room before it came up, and when I came back a lot later it said my kermit window (serial console is nice :)): changing root device to sd0a Enabled INTs: 1, 2, 4, 6, 7, 8, 10, 11, 13, imen: 0x00ffd229 sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0xb sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0xa ahc0: Issued Channel A Bus Reset. 1 SCBs aborted sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0xb6 SEQADDR == 0x3e sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0xb6 SEQADDR == 0x3e ahc0: Issued Channel A Bus Reset. 1 SCBs aborted sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 ahc0: Issued Channel A Bus Reset. 1 SCBs aborted sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 ahc0: Issued Channel A Bus Reset. 1 SCBs aborted sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 ahc0: Issued Channel A Bus Reset. 1 SCBs aborted sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 ahc0: Issued Channel A Bus Reset. 1 SCBs aborted sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 ahc0: Issued Channel A Bus Reset. 1 SCBs aborted sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 SEQADDR == 0x0 ahc0: Issued Channel A Bus Reset. 1 SCBs aborted panic (cpu#0): cannot mount root syncing disks... done Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot <-- Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Sun Dec 1 13:34:00 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA01536 for smp-outgoing; Sun, 1 Dec 1996 13:34:00 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA01526 for ; Sun, 1 Dec 1996 13:33:53 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id OAA15619; Sun, 1 Dec 1996 14:33:40 -0700 Message-Id: <199612012133.OAA15619@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terje Normann Marthinussen cc: freebsd-smp@freebsd.org Subject: Re: SMP-current status In-reply-to: Your message of "Sun, 01 Dec 1996 21:36:41 +0100." <199612012036.VAA10981@slibo.cc.uit.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 01 Dec 1996 14:33:40 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >This might be of a bit more help... > >I let it stay there for a couple of minutes the first time, but it seems >like that wasn't enough, came to reboot again a bit later. Thought I >had another kernel in /, went out of the room before it came up, and when >I came back a lot later it said my kermit window (serial console is nice :)): > >changing root device to sd0a >Enabled INTs: 1, 2, 4, 6, 7, 8, 10, 11, 13, imen: 0x00ffd229 >sd0(ahc0:0:0): timed out while idle, LASTPHASE == 0x1, SCSISIGI == 0x0 smells like a PCI/EISA/level translation problem, I really am growing to *hate* the MP spec!!! - from the output: >ahc0 rev 3 int a irq 7 on pci1:13 ^^^^^ - from your mptable output: I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT active-hi edge 19 0 14 0 INT active-hi edge 19 1 14 1 ... INT conforms level 19 7 14 7 ^^^^^^^^ ^^^^^ ^^ ^ --- without getting into the issues here (different boards do different things and its all a very ugly $#%^&*^*$% mess) looking at your previous mail I'm confused: >FreeBSD 3.0-SMP #0: Sun Dec 1 18:25:33 MET 1996 > terjem@quattro:/usr/src/sys/compile/netserver >FreeBSD/SMP: Multiprocessor motherboard > cpu0 (BSP): apic id: 0, version: 0x00030010 > cpu1 (AP): apic id: 2, version: 0x00030010 > cpu2 (AP): apic id: 3, version: 0x00030010 > cpu3 (AP): apic id: 4, version: 0x00030010 > io0 (APIC): apic id: 14, version: 0x000f0011 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the above line should ONLY come out of an APIC_IO enabled kernel, yet you inply this one isn't when you go on to say: >It then continues to boot, puts #2 into the scheduler and comes up just fine. >I haven't any other OS on that machine right now, but I moved the disk >to another identical machine (btw. it had stepping 11 on CPU 3 and 4) >and got the same results, so I don't think it's the hardware thats >faulty either. > >Tried to compile it with options APIC_IO for the first time as well: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- what is important to establish here is: can you run a 2 CPU system with APIC_IO OFF ("options APIC_IO" NOT in config)? my guess is you can... can you run a 2 CPU system with APIC_IO ON ("options APIC_IO" IS in config)? my guess is you *can't*... -- the safest way to test 2 CPU version for now is to add line to mp_machdep.c: /* start each AP */ + mp_naps = 2; for ( x = mp_naps; x >= 1; --x ) { -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Sun Dec 1 15:33:20 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id PAA06013 for smp-outgoing; Sun, 1 Dec 1996 15:33:20 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA05998 for ; Sun, 1 Dec 1996 15:33:17 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id RAA00322 for ; Sun, 1 Dec 1996 17:33:15 -0600 (CST) Message-Id: <199612012333.RAA00322@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: freebsd-smp@freebsd.org Subject: pmap_enter: modified page not writable... Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 01 Dec 1996 17:33:14 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I get a bunch of these when running the kernel built today. However, everything seems to be working ok. This is with APIC_IO. pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 pmap_remove: modified page not writable: va: 0x44000, pte: 0xee0465 pmap_remove_all: modified page not writable: va: 0x43000, pte: 0xeb4465 Enclosed is an mptable+dmesg fwiw.. Chris =============================================================================== MPTable, version 2.0.4 looking for EBDA pointer @ 0x040e, found, searching EBDA @ 0x0009fc00 searching CMOS 'top of mem' @ 0x0009f800 (638K) searching default 'top of mem' @ 0x0009fc00 (639K) searching BIOS @ 0x000f0000 MP FPS found in BIOS @ physical addr: 0x000f0c80 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000f0c80 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0xf4 mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f0c94 signature: 'PCMP' base table length: 292 version: 1.1 checksum: 0x31 OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 28 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x11 BSP, usable 5 2 1 0x07bf 1 0x11 AP, usable 5 2 1 0x07bf -- Bus: Bus ID Type 0 ISA 1 PCI -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT conforms conforms 0 0 2 0 INT conforms conforms 0 1 2 1 INT conforms conforms 0 0 2 2 INT conforms conforms 0 3 2 3 INT conforms conforms 0 4 2 4 INT conforms conforms 0 5 2 5 INT conforms conforms 0 6 2 6 INT conforms conforms 0 7 2 7 INT conforms conforms 0 8 2 8 INT conforms conforms 0 9 2 9 INT conforms conforms 0 10 2 10 INT conforms conforms 0 11 2 11 INT conforms conforms 0 12 2 12 INT conforms conforms 0 13 2 13 INT conforms conforms 0 14 2 14 INT conforms conforms 0 15 2 15 INT active-lo level 1 8:A 2 16 INT active-lo level 1 9:A 2 17 INT active-lo level 1 10:A 2 18 INT active-lo level 1 12:A 2 19 SMI conforms conforms 0 0 2 23 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT active-hi edge 0 0 255 0 NMI active-hi edge 0 0 255 1 ------------------------------------------------------------------------------- dmesg output: init386 done CR0 = 80000011 Copyright (c) 1992-1996 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.0-SMP #0: Sun Dec 1 17:23:57 CST 1996 root@friley216.res.iastate.edu:/local/sys/compile/SMP FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010 cpu1 (AP): apic id: 1, version: 0x00030010 io0 (APIC): apic id: 2, version: 0x00170011 Calibrating clock(s) relative to mc146818A clock ... i8254 clock: 1193527 Hz CPU: Pentium (586-class CPU) Origin = "GenuineIntel" Id = 0x52c Stepping=12 Features=0x3bf real memory = 33554432 (32768K bytes) avail memory = 30867456 (30144K bytes) Probing for devices on PCI bus 0: chip0 rev 1 on pci0:0 chip1 rev 1 on pci0:7:0 chip2 rev 0 on pci0:7:1 vga0 rev 1 int a irq 18 on pci0:10 Freeing (NOT implimented) irq 11 for ISA cards. ahc0 rev 0 int a irq 19 on pci0:12 Freeing (NOT implimented) irq 9 for ISA cards. ahc0: aic7880 Wide Channel, SCSI Id=7, 16/255 SCBs ahc0: target 0 Tagged Queuing Device (ahc0:0:0): "MICROP 4110-09NB_Nov18F TN0F" type 0 fixed SCSI 2 sd0(ahc0:0:0): Direct-Access 1002MB (2053880 512 byte sectors) Probing for devices on the ISA bus: sc0 at 0x60-0x6f irq 1 on motherboard sc0: VGA color <16 virtual consoles, flags=0x0> sio0 at 0x3f8-0x3ff irq 4 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A sio2 at 0x3e8-0x3ef irq 5 on isa sio2: type 16550A lpt0 at 0x378-0x37f irq 7 on isa lpt0: Interrupt-driven port lp0: TCP/IP capable interface psm0 at 0x60-0x64 irq 12 on motherboard psm0: device ID 0, 3 buttons? fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: NEC 72065B fd0: 1.44MB 3.5in wdc0 at 0x1f0-0x1f7 irq 14 on isa wdc0: unit 0 (wd0): wd0: 408MB (836070 sectors), 899 cyls, 15 heads, 62 S/T, 512 B/S 1 3C5x9 board(s) on ISA found at 0x300 ep0 at 0x300-0x30f irq 10 on isa ep0: aui/utp/bnc[*UTP*] address 00:20:af:75:70:bc npx0 on motherboard npx0: INT 16 interface Enabled INTs: 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 19, imen: 0x00f7aa01 Start pid=2 Start pid=3 Start pid=4 Start pid=5 SMP: Idle procs online, starting an AP! SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... Start pid=6 SMP: TADA! CPU #1 made it into the scheduler!. SMP: All 2 CPU's are online! pmap_enter: modified page not writable: va: 0x17000, pte: 0x847465 pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 pmap_enter: modified page not writable: va: 0x59000, pte: 0x882465 pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 pmap_remove: modified page not writable: va: 0x44000, pte: 0xee0465 pmap_remove_all: modified page not writable: va: 0x43000, pte: 0xeb4465 pmap_remove: modified page not writable: va: 0x44000, pte: 0xee0465 pmap_remove_all: modified page not writable: va: 0x43000, pte: 0x1256465 pmap_enter: modified page not writable: va: 0x2c000, pte: 0x17c5465 pmap_enter: modified page not writable: va: 0x2d000, pte: 0x17c6465 pmap_enter: modified page not writable: va: 0x2f000, pte: 0x18b8465 pmap_enter: modified page not writable: va: 0x30000, pte: 0x18b9465 pmap_enter: modified page not writable: va: 0xefbfd000, pte: 0x1b07465 pmap_remove: modified page not writable: va: 0x52000, pte: 0x1d80465 pmap_enter: modified page not writable: va: 0x27000, pte: 0x199c465 pmap_enter: modified page not writable: va: 0xefbfa000, pte: 0x16c1465 pmap_remove_all: modified page not writable: va: 0xefbfd000, pte: 0x1506465 pmap_enter: modified page not writable: va: 0x35000, pte: 0xfe9465 pmap_enter: modified page not writable: va: 0x27000, pte: 0x1b74465 pmap_enter: modified page not writable: va: 0x68000, pte: 0x11d2465 ------------------------------------------------------------------------------- # SMP kernel config file options: options SMP # Symmetric MultiProcessor Kernel #options APIC_IO # Symmetric (APIC) I/O options NCPU=2 # number of CPUs options NBUS=2 # number of busses options NAPIC=1 # number of IO APICs options NINTR=21 # number of INTs =============================================================================== From owner-freebsd-smp Sun Dec 1 16:06:25 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA07743 for smp-outgoing; Sun, 1 Dec 1996 16:06:25 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id QAA07738 for ; Sun, 1 Dec 1996 16:06:22 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA16369; Sun, 1 Dec 1996 17:06:14 -0700 Message-Id: <199612020006.RAA16369@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Chris Csanady cc: freebsd-smp@freebsd.org Subject: Re: pmap_enter: modified page not writable... In-reply-to: Your message of "Sun, 01 Dec 1996 17:33:14 CST." <199612012333.RAA00322@friley216.res.iastate.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 01 Dec 1996 17:06:14 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >I get a bunch of these when running the kernel built today. However, >everything seems to be working ok. This is with APIC_IO. > >pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 >pmap_remove: modified page not writable: va: 0x44000, pte: 0xee0465 >pmap_remove_all: modified page not writable: va: 0x43000, pte: 0xeb4465 this is new.. try disabling the IDE controller in the BIOS. try a non APIC_IO version. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Sun Dec 1 17:31:53 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA11547 for smp-outgoing; Sun, 1 Dec 1996 17:31:53 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA11542 for ; Sun, 1 Dec 1996 17:31:49 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.3/8.8.3) with ESMTP id JAA11337; Mon, 2 Dec 1996 09:31:27 +0800 (WST) Message-Id: <199612020131.JAA11337@spinner.DIALix.COM> To: Steve Passe cc: Chris Csanady , freebsd-smp@freebsd.org Subject: Re: pmap_enter: modified page not writable... In-reply-to: Your message of "Sun, 01 Dec 1996 17:06:14 MST." <199612020006.RAA16369@clem.systemsix.com> Date: Mon, 02 Dec 1996 09:31:26 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve Passe wrote: > Hi, > > >I get a bunch of these when running the kernel built today. However, > >everything seems to be working ok. This is with APIC_IO. > > > >pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 > >pmap_remove: modified page not writable: va: 0x44000, pte: 0xee0465 > >pmap_remove_all: modified page not writable: va: 0x43000, pte: 0xeb4465 > > this is new.. I'm pretty sure this is concrete evidence that the lack of tlb flushing is causing problems... Cheers, -Peter From owner-freebsd-smp Sun Dec 1 17:50:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA12285 for smp-outgoing; Sun, 1 Dec 1996 17:50:24 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA12280 for ; Sun, 1 Dec 1996 17:50:14 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.3/8.8.3) with ESMTP id JAA11506; Mon, 2 Dec 1996 09:49:33 +0800 (WST) Message-Id: <199612020149.JAA11506@spinner.DIALix.COM> To: Steve Passe cc: Chris Csanady , freebsd-smp@freebsd.org Subject: Re: pmap_enter: modified page not writable... In-reply-to: Your message of "Sun, 01 Dec 1996 17:06:14 MST." <199612020006.RAA16369@clem.systemsix.com> Date: Mon, 02 Dec 1996 09:49:32 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve Passe wrote: > Hi, > > >I get a bunch of these when running the kernel built today. However, > >everything seems to be working ok. This is with APIC_IO. > > > >pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 > >pmap_remove: modified page not writable: va: 0x44000, pte: 0xee0465 > >pmap_remove_all: modified page not writable: va: 0x43000, pte: 0xeb4465 > > this is new.. Just as a BTW, the fix for this *requires* APIC_IO.. Once we've got it working without problems, APIC_IO will be pretty much be mandatory.. Cheers, -Peter From owner-freebsd-smp Sun Dec 1 21:12:16 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id VAA01807 for smp-outgoing; Sun, 1 Dec 1996 21:12:16 -0800 (PST) Received: from mystic.sedona.net (hogan@mystic.sedona.net [204.157.202.250]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id VAA01802 for ; Sun, 1 Dec 1996 21:12:14 -0800 (PST) Received: (from hogan@localhost) by mystic.sedona.net (8.8.3/8.8.HW) id WAA02550 for freebsd-smp@freebsd.org; Sun, 1 Dec 1996 22:08:03 -0700 (MST) From: Hogan Whittall Message-Id: <199612020508.WAA02550@mystic.sedona.net> Subject: Still system slowdowns with certain apps To: freebsd-smp@freebsd.org Date: Sun, 1 Dec 1996 22:08:03 -0700 (MST) X-Mailer: ELM [version 2.4ME+ PL28 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I've got a Tyan TomCat (2) 150MHz with a 3940U controller, Stealth video, and SMC ISA ethernet card using the SMP kernel with APIC_IO undefined (won't work with it defined. make depend fails) and IGNORE_IDLEPROCS commented out of smptests.h...my problem is this... The new kernel works great and I've yet to have a program seg fault on me...but ever since I got the latest source on Thursday night I've experienced a major system slowdown whenever I compile something. The compile itself moves right along and gets done fast, but everything else on the system takes forever to respond until the compile is done. Before Thursday everything worked fine whencompiling and there were still seg faults...but this slowdown is new and I was hoping code I got today would fix it...but it's still there. Has anyone else experienced this or know of a fix? Also, in the proc list... root 5 99.2 0.0 0 12 ?? RL 31Dec69 0:00.00 (cpuidle0) root 6 0.0 0.0 0 12 ?? RL 31Dec69 0:00.00 (cpuidle1) The 2 things I notice are broken start times and the % cpu being used by these. In the past they've always used about 49% each but now with this kernel the majority is on one. Ideas? -- _-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_ | Hogan Whittall Systems & Networking Technician | | hogan@sedona.net Sedona Internet Services, Inc. | | http://cloofone.sedona.net/ Ph: (520) 204-2247 Pg: 282-8030 | |-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-| From owner-freebsd-smp Sun Dec 1 22:04:23 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id WAA03868 for smp-outgoing; Sun, 1 Dec 1996 22:04:23 -0800 (PST) Received: from dyson.iquest.net ([198.70.144.127]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id WAA03863 for ; Sun, 1 Dec 1996 22:04:20 -0800 (PST) Received: (from root@localhost) by dyson.iquest.net (8.8.2/8.6.9) id BAA04106; Mon, 2 Dec 1996 01:03:52 -0500 (EST) From: "John S. Dyson" Message-Id: <199612020603.BAA04106@dyson.iquest.net> Subject: Re: pmap_enter: modified page not writable... To: peter@spinner.dialix.com (Peter Wemm) Date: Mon, 2 Dec 1996 01:03:52 -0500 (EST) Cc: smp@csn.net, ccsanady@friley216.res.iastate.edu, freebsd-smp@freebsd.org In-Reply-To: <199612020131.JAA11337@spinner.DIALix.COM> from "Peter Wemm" at Dec 2, 96 09:31:26 am X-Mailer: ELM [version 2.4 PL24 ME8] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > Steve Passe wrote: > > Hi, > > > > >I get a bunch of these when running the kernel built today. However, > > >everything seems to be working ok. This is with APIC_IO. > > > > > >pmap_enter: modified page not writable: va: 0x58000, pte: 0x881465 > > >pmap_remove: modified page not writable: va: 0x44000, pte: 0xee0465 > > >pmap_remove_all: modified page not writable: va: 0x43000, pte: 0xeb4465 > > > > this is new.. > > I'm pretty sure this is concrete evidence that the lack of tlb flushing > is causing problems... > I'll help you all check things out tomorrow (time to go to bed), if you still need help by then. John From owner-freebsd-smp Sun Dec 1 22:14:50 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id WAA04278 for smp-outgoing; Sun, 1 Dec 1996 22:14:50 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id WAA04271 for ; Sun, 1 Dec 1996 22:14:48 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id XAA18197; Sun, 1 Dec 1996 23:14:37 -0700 Message-Id: <199612020614.XAA18197@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Hogan Whittall cc: freebsd-smp@freebsd.org Subject: Re: Still system slowdowns with certain apps In-reply-to: Your message of "Sun, 01 Dec 1996 22:08:03 MST." <199612020508.WAA02550@mystic.sedona.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 01 Dec 1996 23:14:37 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > I've got a Tyan TomCat (2) 150MHz with a 3940U controller, Stealth video, > and SMC ISA ethernet card using the SMP kernel with APIC_IO undefined > (won't work with it defined. make depend fails) and > IGNORE_IDLEPROCS commented out of smptests.h...my problem is this... > ... I now believe that SMP-current is more broken than we previously thought. My recommendation is to NOT use the post-merge SMP kernel for *anything* till we figure it out! -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Dec 2 02:16:55 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA14366 for smp-outgoing; Mon, 2 Dec 1996 02:16:55 -0800 (PST) Received: from zwei.siemens.at (zwei.siemens.at [193.81.246.12]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id CAA14361 for ; Mon, 2 Dec 1996 02:16:50 -0800 (PST) Received: from sol1.gud.siemens.co.at (root@[10.1.143.100]) by zwei.siemens.at (8.7.5/8.7.3) with SMTP id LAA15986 for ; Mon, 2 Dec 1996 11:16:08 +0100 (MET) Received: from ws2301.gud.siemens.co.at by sol1.gud.siemens.co.at with smtp (Smail3.1.28.1 #7 for ) id m0vUVPx-0001y9C; Mon, 2 Dec 96 11:16 MET Received: by ws2301.gud.siemens.co.at (1.37.109.16/1.37) id AA243241632; Mon, 2 Dec 1996 11:13:52 +0100 From: "Hr.Ladavac" Message-Id: <199612021013.AA243241632@ws2301.gud.siemens.co.at> Subject: Re: Pentium CPU steppings To: phk@critter.tfs.com (Poul-Henning Kamp) Date: Mon, 2 Dec 1996 11:13:52 +0100 (MEZ) Cc: peter@spinner.dialix.com, freebsd-smp@freebsd.org In-Reply-To: <3057.849292563@critter.tfs.com> from "Poul-Henning Kamp" at Nov 29, 96 07:36:03 pm X-Mailer: ELM [version 2.4 PL24 ME8a] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk E-mail message from Poul-Henning Kamp contained: > > In message <199611291828.CAA19491@spinner.DIALix.COM>, Peter Wemm writes: > >Poul-Henning Kamp wrote: > >> >Hmm, I wonder if it's safe to test for this at the end of npx.c in the > >> >kernel? > >> > >> Linux can do it, so why couldn't we ? > > > >Yeah, but have you ever seen it report "yes"? Last I checked, (2.0.twenty > >something), the code looked very much like it hardwired the "fdiv bug > >present?" result to "no" and I could not find any code in the kernel that > > yes, they circumvent it, I belive in their compiler :-) But do they circumvent it in all instances? According to c't magazine, Intel admitted that some other operations that internally use FDIV are also buggy. From the top of my head: ATAN ATAN2 and about 5 or 6 others. N.B. the interpolation trick does not help at all if you use these (and practically all of ray-tracing software does ): /Marino > > -- > Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. > http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. > whois: [PHK] | phk@ref.tfs.com TRW Financial Systems, Inc. > Future will arrive by its own means, progress not so. > From owner-freebsd-smp Mon Dec 2 02:35:20 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA14875 for smp-outgoing; Mon, 2 Dec 1996 02:35:20 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id CAA14870 for ; Mon, 2 Dec 1996 02:35:14 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id LAA20262; Mon, 2 Dec 1996 11:35:06 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id LAA10720; Mon, 2 Dec 1996 11:35:05 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id LAA07875; Mon, 2 Dec 1996 11:35:00 +0100 (MET) Message-Id: <199612021035.LAA07875@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Steve Passe cc: freebsd-smp@freebsd.org Subject: Re: SMP-current status In-reply-to: Your message of "Sun, 01 Dec 1996 14:33:40 MET." <199612012133.OAA15619@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 11:34:59 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >smells like a PCI/EISA/level translation problem, >I really am growing to *hate* the MP spec!!! > >- >from the output: >>ahc0 rev 3 int a irq 7 on pci1:13 > ^^^^^ >- >from your mptable output: > >I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# > ExtINT active-hi edge 19 0 14 0 > INT active-hi edge 19 1 14 1 > ... > INT conforms level 19 7 14 7 > ^^^^^^^^ ^^^^^ ^^ ^ >--- >without getting into the issues here (different boards do different things >and its all a very ugly $#%^&*^*$% mess) Yes, I noticed a couple of days after I first sent in my mptable output and you commented that nothing seemed to be on the XPRESS bus, I looked on the mptable output again, and it stroke me that nothing seemed to be on the PCI bus either, even though there should be at least a 3COM ethernet card and two SCSI controllers. > >looking at your previous mail I'm confused: > >>FreeBSD 3.0-SMP #0: Sun Dec 1 18:25:33 MET 1996 >> terjem@quattro:/usr/src/sys/compile/netserver >>FreeBSD/SMP: Multiprocessor motherboard >> cpu0 (BSP): apic id: 0, version: 0x00030010 >> cpu1 (AP): apic id: 2, version: 0x00030010 >> cpu2 (AP): apic id: 3, version: 0x00030010 >> cpu3 (AP): apic id: 4, version: 0x00030010 >> io0 (APIC): apic id: 14, version: 0x000f0011 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > >the above line should ONLY come out of an APIC_IO enabled kernel, yet you >inply this one isn't when you go on to say: Ai... My error. Cut&Paste from the wrong boot, should have been: FreeBSD 3.0-SMP #0: Sun Dec 1 17:30:48 MET 1996 terjem@quattro:/usr/src/sys/compile/netserver FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010 cpu1 (AP): apic id: 2, version: 0x00030010 cpu2 (AP): apic id: 3, version: 0x00030010 cpu3 (AP): apic id: 4, version: 0x00030010 Warning: APIC I/O disabled >what is important to establish here is: > > can you run a 2 CPU system with APIC_IO OFF ("options APIC_IO" NOT in config)? > my guess is you can... > > > can you run a 2 CPU system with APIC_IO ON ("options APIC_IO" IS in config)? > my guess is you *can't*... > You are guessing right. >the safest way to test 2 CPU version for now is to add line to mp_machdep.c: > > /* start each AP */ >+ mp_naps = 2; > for ( x = mp_naps; x >= 1; --x ) { > A bit wrong there. Should be mp_naps = 1; Only 1 application processor in a 2 CPU system. It's the way I first booted 2 CPUs. Doesn't help though. Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Mon Dec 2 02:50:43 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA15168 for smp-outgoing; Mon, 2 Dec 1996 02:50:43 -0800 (PST) Received: from hermes.lsi.usp.br ([143.107.161.220]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id CAA15163 for ; Mon, 2 Dec 1996 02:50:40 -0800 (PST) Received: from sofia (sofia.lsi.usp.br [143.107.161.141]) by hermes.lsi.usp.br (8.7.5/8.7.3) with SMTP id IAA23590; Mon, 2 Dec 1996 08:49:10 -0200 (BDB) Date: Mon, 2 Dec 1996 07:49:58 -0500 (EST) From: Mario Donato Marino X-Sender: mario@sofia To: Erich Boleyn cc: Steve Passe , smp@freebsd.org Subject: Re: SMP-current hang problems. In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Sat, 30 Nov 1996, Erich Boleyn wrote: > > Steve Passe writes: > > > we have decided to more or less freeze development till we figure out > > the "system hang" problem with the latest code. If you have one > > of the systems that hangs anywhere in the area of: > > > > SMP: Idle procs online, starting an AP! > > SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... > > SMP: TADA! CPU #1 made it into the scheduler!. > > SMP: All 2 CPU's are online! > > ... > > Well, I wanted to try an unmunged syscons, so I took the existing > SMP source tree and tried it with and without APIC_IO. (My test box is > a 4-CPU Pentium Pro with PCI SCSI and EISA network cards) What is your machine ? Is it a AMI Goliath ? > > Without APIC_IO, I got messages like the above for all 3 of the other CPUs > in my box. Pretty cool! Syscons was screwed up here, so I don't think > it is just an "APIC_IO" problem. I could bang on the keyboard for a long > time and it kept going. > > With APIC_IO, I got one message like the above for "CPU #3", but after > the "Starting Scheduling..." part, it said "SMP: freezing CPU #3", then > no more SMP CPU booting messages. Syscons was screwed up here in > apparently the exact same fashion. If I banged on the keyboard too much, > it would lock up. > > I think I'm more interested in fixing syscons than regressing it and > a bunch of other files, perhaps with unpredictable results. > > More later. (is there anything specific I should try?) > > -- > Erich Stefan Boleyn \_ E-mail (preferred): > Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) > Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" > From owner-freebsd-smp Mon Dec 2 02:59:51 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA15346 for smp-outgoing; Mon, 2 Dec 1996 02:59:51 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id CAA15341 for ; Mon, 2 Dec 1996 02:59:48 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id DAA19667 for ; Mon, 2 Dec 1996 03:59:45 -0700 Message-Id: <199612021059.DAA19667@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: freebsd-smp@freebsd.org Subject: System hang found (I think) Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 03:59:44 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, Good news, I have been recreating every kernel since the last merge and have tracked the damage down to code submitted on 96-11-25. So if you checkout a tree of code immediately previous to that date you should be in reasonable shape. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Dec 2 03:24:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16023 for smp-outgoing; Mon, 2 Dec 1996 03:24:01 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16004 for freebsd-smp; Mon, 2 Dec 1996 03:23:57 -0800 (PST) Date: Mon, 2 Dec 1996 03:23:57 -0800 (PST) From: Peter Wemm Message-Id: <199612021123.DAA16004@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 swtch.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 03:23:56 Modified: i386/i386 swtch.s Log: Conditionalise the per-cpu private pages support so that it can be disabled. Steve has identified this as the prime suspect for the lockups and sig11's. (more commits to come) Revision Changes Path 1.28 +3 -3 sys/i386/i386/swtch.s From owner-freebsd-smp Mon Dec 2 03:26:15 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16113 for smp-outgoing; Mon, 2 Dec 1996 03:26:15 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16104 for freebsd-smp; Mon, 2 Dec 1996 03:26:13 -0800 (PST) Date: Mon, 2 Dec 1996 03:26:13 -0800 (PST) From: Peter Wemm Message-Id: <199612021126.DAA16104@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 locore.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 03:26:12 Modified: i386/i386 locore.s Log: Conditionalise the per-cpu private pages code Revision Changes Path 1.32 +8 -8 sys/i386/i386/locore.s From owner-freebsd-smp Mon Dec 2 03:30:58 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16262 for smp-outgoing; Mon, 2 Dec 1996 03:30:58 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16255 for freebsd-smp; Mon, 2 Dec 1996 03:30:56 -0800 (PST) Date: Mon, 2 Dec 1996 03:30:56 -0800 (PST) From: Peter Wemm Message-Id: <199612021130.DAA16255@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/include pmap.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 03:30:55 Modified: i386/include pmap.h Log: Conditionalise the per-cpu private pages code. (I think I see what the problem might have been.. lots of code #includes pmap.h, but this was now conditional on #define SMP, and I'll bet that some code had not included "opt_smp.h" first. This would have caused two different values for NKPDE etc to be in use in the kernel) Revision Changes Path 1.3 +7 -7 sys/i386/include/pmap.h From owner-freebsd-smp Mon Dec 2 03:34:38 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16437 for smp-outgoing; Mon, 2 Dec 1996 03:34:38 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA16426 for freebsd-smp; Mon, 2 Dec 1996 03:34:36 -0800 (PST) Date: Mon, 2 Dec 1996 03:34:36 -0800 (PST) From: Peter Wemm Message-Id: <199612021134.DAA16426@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/conf options.i386 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 03:34:34 Modified: i386/conf options.i386 Log: List the (undocumented) option to enable the private pages code Revision Changes Path 1.11 +1 -0 sys/i386/conf/options.i386 From owner-freebsd-smp Mon Dec 2 07:23:39 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA25549 for smp-outgoing; Mon, 2 Dec 1996 07:23:39 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA25539 for freebsd-smp; Mon, 2 Dec 1996 07:23:36 -0800 (PST) Date: Mon, 2 Dec 1996 07:23:36 -0800 (PST) From: Peter Wemm Message-Id: <199612021523.HAA25539@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 serial.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 07:23:35 Added: i386/i386 serial.s Log: ressurect serial.s From owner-freebsd-smp Mon Dec 2 11:23:53 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA07187 for smp-outgoing; Mon, 2 Dec 1996 11:23:53 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA07179; Mon, 2 Dec 1996 11:23:47 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id MAA22076; Mon, 2 Dec 1996 12:23:45 -0700 Message-Id: <199612021923.MAA22076@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Peter Wemm cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of "Mon, 02 Dec 1996 03:30:56 PST." <199612021130.DAA16255@freefall.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 12:23:44 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, > Modified: i386/include pmap.h > ... > (I think I see what the problem might have been.. lots of code #includes > pmap.h, but this was now conditional on #define SMP, and I'll bet that > some code had not included "opt_smp.h" first. This would have caused > two different values for NKPDE etc to be in use in the kernel) That would explain alot!!! I think as a first step in the merging of SMP with -current we might add the line: #include "opt_smp" to every file that now contains SMP code, At that point everyone would have a good idea of where we will be affecting the -current tree. We probably should also document all SMP specific options in conf/LINT. The next step would be to break up files that have grown to have alot of SMP stuff mixed with standard stuff, eg. init_main.c -> init_main.c & init_smp.c. Stated another way, I think we should strive to keep SMP changes localized to SMP specific files to the extent that it is practical. We also need to think thru the locations of some things b4 the merge: does "options SMP" belong in sys/i386/conf/options.i386 or sys/conf/options? "options APIC_IO" probably does, can options from options be mixed with options in options.i386, ie can SMP be in one, APIC_IO in the other, and have them both end up in "opt_smp"? We need to start collecting docs, should that be in: src/share/doc, which would be outside of sys, or should we creat sys/SMP/doc, or ??? We also need an SMP specific 'utility' directory for things like mptable.c, sys/SMP/util perhaps? -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Dec 2 11:55:37 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA08917 for smp-outgoing; Mon, 2 Dec 1996 11:55:37 -0800 (PST) Received: from bacall.lodgenet.com (bacall.lodgenet.com [205.138.147.242]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA08911 for ; Mon, 2 Dec 1996 11:55:24 -0800 (PST) Received: (from mail@localhost) by bacall.lodgenet.com (8.6.12/8.6.12) id NAA10513 for ; Mon, 2 Dec 1996 13:54:51 -0600 Received: from garbo.lodgenet.com(204.124.123.250) by bacall via smap (V1.3) id sma010502; Mon Dec 2 13:54:34 1996 Received: from jake.lodgenet.com (jake.lodgenet.com [10.0.11.30]) by garbo.lodgenet.com (8.6.12/8.6.9) with ESMTP id NAA17340 for ; Mon, 2 Dec 1996 13:54:38 -0600 Received: from jake.lodgenet.com (localhost [127.0.0.1]) by jake.lodgenet.com (8.8.3/8.6.12) with ESMTP id NAA04298 for ; Mon, 2 Dec 1996 13:49:28 -0600 (CST) Message-Id: <199612021949.NAA04298@jake.lodgenet.com> X-Mailer: exmh version 1.6.9 8/22/96 To: smp@freebsd.org Subject: Status report... Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 13:49:28 -0600 From: "Eric L. Hernes" Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Howdy, I just tested a kernel from this morning (12/1). It is *much* better than the one I tried last week. Last week I got lots of sig11's, but I haven't seen one yet! But I am getting a few silo overflows from the mouse, dmesg shows: sio1: 1 more silo overflow (total 1) sio1: 2 more silo overflows (total 3) sio1: 1 more silo overflow (total 4) sio1: 1 more silo overflow (total 5) sio1: 1 more silo overflow (total 6) sio1: 1 more silo overflow (total 7) sio1: 1 more silo overflow (total 8) sio1: 1 more silo overflow (total 9) sio1: 209 more interrupt-level buffer overflows (total 209) sio1: 1 more silo overflow (total 10) sio1: 1 more silo overflow (total 11) sio1: 446 more interrupt-level buffer overflows (total 655) ed0: device timeout ed0: device timeout ed0: device timeout ed0: device timeout ed0: device timeout sio1: 47 more interrupt-level buffer overflows (total 702) sio1: 1 more silo overflow (total 12) sio1: 1 more silo overflow (total 13) sio1: 1 more silo overflow (total 14) which are probably all interrupt related, no? These only show up under load, like `make -j8', but it's definately useable! good work! eric. -- erich@lodgenet.com http://rrnet.com/~erich erich@rrnet.com From owner-freebsd-smp Mon Dec 2 12:16:11 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA10241 for smp-outgoing; Mon, 2 Dec 1996 12:16:11 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA10232 for ; Mon, 2 Dec 1996 12:16:08 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id NAA22405; Mon, 2 Dec 1996 13:16:00 -0700 Message-Id: <199612022016.NAA22405@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: "Eric L. Hernes" cc: smp@freebsd.org Subject: Re: Status report... In-reply-to: Your message of "Mon, 02 Dec 1996 13:49:28 CST." <199612021949.NAA04298@jake.lodgenet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 13:16:00 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > I just tested a kernel from this morning (12/1). It is *much* better > than the one I tried last week. Last week I got lots of sig11's, but > I haven't seen one yet! But I am getting a few silo overflows > from the mouse, dmesg shows: > > sio1: 1 more silo overflow (total 1) > ... > sio1: 209 more interrupt-level buffer overflows (total 209) > ... > ed0: device timeout > ... > which are probably all interrupt related, no? These only show up > under load, like `make -j8', but it's definately useable! I'm in the middle of testing the various flavors right now. I got several "de0 device timeout"s, have never seen these b4. This was with APIC_IO on, am about to test non APIC_IO version. Sounds INTerrupt realted, but I haven't had a chance to look at it yet... -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Dec 2 12:30:07 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA11030 for smp-outgoing; Mon, 2 Dec 1996 12:30:07 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA11024; Mon, 2 Dec 1996 12:30:05 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id OAA04003; Mon, 2 Dec 1996 14:27:38 -0600 (CST) Message-Id: <199612022027.OAA04003@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: Steve Passe cc: Peter Wemm , freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of Mon, 02 Dec 1996 12:23:44 -0700. <199612021923.MAA22076@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 14:27:38 -0600 From: Chris Csanady Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk >I think as a first step in the merging of SMP with -current we might >add the line: #include "opt_smp" to every file that now contains SMP code, >At that point everyone would have a good idea of where we will be >affecting the -current tree. > >We probably should also document all SMP specific options in conf/LINT. Sounds good. Although, as we progress, this may be almost all the files in /sys. I was hoping to start moving the locking out into the syscalls. >The next step would be to break up files that have grown to have alot >of SMP stuff mixed with standard stuff, eg. init_main.c -> init_main.c & >init_smp.c. Stated another way, I think we should strive to keep SMP changes >localized to SMP specific files to the extent that it is practical. For now perhaps. But wouldn't the end goal be to have the UP instance just 1 processor SMP case? Im thinking that this would make the code cleaner, and more generic in the end. I think having kernel threads would be well worth it. --Chris Csanady >We also need to think thru the locations of some things b4 the merge: > > does "options SMP" belong in sys/i386/conf/options.i386 or sys/conf/options? > "options APIC_IO" probably does, can options from options be mixed with > options in options.i386, ie can SMP be in one, APIC_IO in the other, and > have them both end up in "opt_smp"? > >We need to start collecting docs, should that be in: src/share/doc, >which would be outside of sys, or should we creat sys/SMP/doc, or ??? > >We also need an SMP specific 'utility' directory for things like mptable.c, >sys/SMP/util perhaps? >-- >Steve Passe | powered by >smp@csn.net | FreeBSD > From owner-freebsd-smp Mon Dec 2 12:44:29 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA12209 for smp-outgoing; Mon, 2 Dec 1996 12:44:29 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA12201 for ; Mon, 2 Dec 1996 12:44:26 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id NAA22589; Mon, 2 Dec 1996 13:44:18 -0700 Message-Id: <199612022044.NAA22589@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Chris Csanady cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of "Mon, 02 Dec 1996 14:27:38 CST." <199612022027.OAA04003@friley216.res.iastate.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 13:44:18 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, > >The next step would be to break up files that have grown to have alot > >of SMP stuff mixed with standard stuff, eg. init_main.c -> init_main.c & > >init_smp.c. Stated another way, I think we should strive to keep SMP changes > >localized to SMP specific files to the extent that it is practical. > > For now perhaps. But wouldn't the end goal be to have the UP instance just > 1 processor SMP case? Im thinking that this would make the code cleaner, and > more generic in the end. I think having kernel threads would be well worth > it. Long term, I think I agree, I go back and forth on just what the model should look like. For the short term I have many reasons for my "minimal impact" model: making an SMP kernel that boots & runs on a uniprocessor board is currently impossible for a lot of 'architectural' reasons, ie alot would have to change in the mainline code to make this work. Look at how we pervert things like curproc for an example. such a model allows us to arrive at a starting point from which everyone can view what is done to support SMP, without changing the code base in which they feel comfortable. we don't upset -current devvelopers who are unconcerned about SMP by changing the areas they are working on. we don't bog down in 'political' battles about what should change and why. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Dec 2 13:05:51 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA13377 for smp-outgoing; Mon, 2 Dec 1996 13:05:51 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA13365 for ; Mon, 2 Dec 1996 13:05:47 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id NAA11277; Mon, 2 Dec 1996 13:47:07 -0700 From: Terry Lambert Message-Id: <199612022047.NAA11277@phaeton.artisoft.com> Subject: Re: cvs commit: sys/i386/include pmap.h To: smp@csn.net (Steve Passe) Date: Mon, 2 Dec 1996 13:47:06 -0700 (MST) Cc: ccsanady@friley216.res.iastate.edu, freebsd-smp@freefall.freebsd.org In-Reply-To: <199612022044.NAA22589@clem.systemsix.com> from "Steve Passe" at Dec 2, 96 01:44:18 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > Long term, I think I agree, I go back and forth on just what the model > should look like. For the short term I have many reasons for my > "minimal impact" model: [ ... expedient reasons ... ] > making an SMP kernel that boots & runs on a uniprocessor board is currently > impossible for a lot of 'architectural' reasons, ie alot would have to change > in the mainline code to make this work. Look at how we pervert things > like curproc for an example. The main line code should be changed, regardless. It would definitely benefit from the changes, and the interrupt architectures for non-Intel machines dictate a similar change be made for porting in any case. I would like to see the change driven by the SMP requirements, since it would be much harder to go back and retrofit SMP if the changes only considered the porting requirements, and not the SMP. The SMP code integration is going to be A Big Deal. If you can fit some additional (good!) changes in under that umbrella, so much the better. Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Dec 2 13:25:26 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA14915 for smp-outgoing; Mon, 2 Dec 1996 13:25:26 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA14908 for ; Mon, 2 Dec 1996 13:25:22 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id OAA22855; Mon, 2 Dec 1996 14:23:53 -0700 Message-Id: <199612022123.OAA22855@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: ccsanady@friley216.res.iastate.edu, freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of "Mon, 02 Dec 1996 13:47:06 MST." <199612022047.NAA11277@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 14:23:53 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > > Long term, I think I agree, I go back and forth on just what the model > > ... > [ ... expedient reasons ... ] > > ... > > The main line code should be changed, regardless. It would definitely > benefit from the changes, and the interrupt architectures for non-Intel > machines dictate a similar change be made for porting in any case. > > I would like to see the change driven by the SMP requirements, since > it would be much harder to go back and retrofit SMP if the changes > only considered the porting requirements, and not the SMP. > > The SMP code integration is going to be A Big Deal. If you can fit > some additional (good!) changes in under that umbrella, so much the > better. all valid points. I'm not arguing against this, only about when... look at it from this point of view. Peter currently merges SMP with -current every 4-8 weeks. At such a point in time we could call SMP-current '-current' and the transition would be complete. That is more or less how I see SMP getting into -current, with it being several steps as I outlined previously. If we start doing all the admittedly desirable changes as part of the merge process we will be burning much too much energy staying in sync with -current while we do it. I want: minor changes to SMP-current \ +-- start necessary long term changes. -current / Not: fix SMP-current, fix SMP-current, ... fix SMP-current \ +-- someday... fix -current, fix -current, ... fix -current / -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Dec 2 13:33:19 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA15354 for smp-outgoing; Mon, 2 Dec 1996 13:33:19 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id NAA15343 for ; Mon, 2 Dec 1996 13:33:12 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id OAA22923 for ; Mon, 2 Dec 1996 14:33:09 -0700 Message-Id: <199612022133.OAA22923@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: freebsd-smp@freefall.freebsd.org Subject: Re: Status report... In-reply-to: Your message of "Mon, 02 Dec 1996 13:16:00 MST." <199612022016.NAA22405@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 14:33:09 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, > > I just tested a kernel from this morning (12/1). It is *much* better > > than the one I tried last week. Last week I got lots of sig11's, but > > I haven't seen one yet! But I am getting a few silo overflows > > from the mouse, dmesg shows: > > > > sio1: 1 more silo overflow (total 1) > > ... > > sio1: 209 more interrupt-level buffer overflows (total 209) > > ... > > ed0: device timeout > > ... > > which are probably all interrupt related, no? These only show up > > under load, like `make -j8', but it's definately useable! > > I'm in the middle of testing the various flavors right now. > I got several "de0 device timeout"s, have never seen these b4. > This was with APIC_IO on, am about to test non APIC_IO version. > Sounds INTerrupt realted, but I haven't had a chance to look at it > yet... the non APIC_IO version seems to work fine. The APIC_IO versions is definately loosing INTs, so I'm off to chase that next. I've rebuit both flavors several times now, and have seen NO sigs/segfaults!!! --- ok, testers, the balls in your court now,get the latest code and check it out. - the score card so far: Peter's working. I'm working (lost INTs in APIC_IO version) Eric L. Herne working (lost INTs in APIC_IO version) -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Dec 2 14:03:48 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA17016 for smp-outgoing; Mon, 2 Dec 1996 14:03:48 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id OAA17010 for ; Mon, 2 Dec 1996 14:03:44 -0800 (PST) Received: from critter.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0vUgS4-0003xRC; Mon, 2 Dec 96 14:03 PST Received: from critter.tfs.com (localhost.phk.dk [127.0.0.1]) by critter.tfs.com (8.8.2/8.8.2) with ESMTP id XAA03015; Mon, 2 Dec 1996 23:04:21 +0100 (MET) To: Steve Passe cc: Chris Csanady , freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of "Mon, 02 Dec 1996 13:44:18 MST." <199612022044.NAA22589@clem.systemsix.com> Date: Mon, 02 Dec 1996 23:04:20 +0100 Message-ID: <3013.849564260@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk In message <199612022044.NAA22589@clem.systemsix.com>, Steve Passe writes: >Hi, > >> >The next step would be to break up files that have grown to have alot >> >of SMP stuff mixed with standard stuff, eg. init_main.c -> init_main.c & >> >init_smp.c. Stated another way, I think we should strive to keep SMP chang >es >> >localized to SMP specific files to the extent that it is practical. >> >> For now perhaps. But wouldn't the end goal be to have the UP instance just >> 1 processor SMP case? Im thinking that this would make the code cleaner, an >d >> more generic in the end. I think having kernel threads would be well worth >> it. > >Long term, I think I agree, I go back and forth on just what the model >should look like. For the short term I have many reasons for my >"minimal impact" model: Steve, this is not really a choice. SMP will not really be an option, since a lot of interfaces to the kernel (curproc being but one) will change. People will have LKM's accesing the wrong kind of lkm and it will be a proper mess. We are going to make the change once, and we're not going to have two different versions of those interfaces around (IMO) I would also say that I think it would be a good idea if you started to look at merging the smp stuff into the lite2 tree, since the first merge that will happen is probably the lite2 tree into current. That said, I think that it is correct that we need to think and decide on a structure and layout of SMP versus UP kernel. In particular somebody should think VERY hard about what bits are MI and what are MD bits in the SMP stuff. -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Mon Dec 2 14:14:55 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA17592 for smp-outgoing; Mon, 2 Dec 1996 14:14:55 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id OAA17587 for ; Mon, 2 Dec 1996 14:14:52 -0800 (PST) Received: from critter.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0vUgd1-0003yVC; Mon, 2 Dec 96 14:14 PST Received: from critter.tfs.com (localhost.phk.dk [127.0.0.1]) by critter.tfs.com (8.8.2/8.8.2) with ESMTP id XAA03050; Mon, 2 Dec 1996 23:15:55 +0100 (MET) To: Steve Passe cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of "Mon, 02 Dec 1996 14:23:53 MST." <199612022123.OAA22855@clem.systemsix.com> Date: Mon, 02 Dec 1996 23:15:55 +0100 Message-ID: <3048.849564955@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > >I want: > > minor changes to SMP-current \ > +-- start necessary long term changes. > -current / > >Not: > > fix SMP-current, fix SMP-current, ... fix SMP-current \ > +-- someday... > fix -current, fix -current, ... fix -current / > Well, I think you can have: smp --- \ lite2 --------- \ current-----------------> or instead smp -------------- \ lite2 --------- \ \ \ current-----------------> But I'm pretty sure that lite2 will get merged before smp as things stand now. -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Mon Dec 2 14:20:28 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA17945 for smp-outgoing; Mon, 2 Dec 1996 14:20:28 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id OAA17936 for ; Mon, 2 Dec 1996 14:20:25 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id PAA23260; Mon, 2 Dec 1996 15:20:16 -0700 Message-Id: <199612022220.PAA23260@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Poul-Henning Kamp cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of "Mon, 02 Dec 1996 23:15:55 +0100." <3048.849564955@critter.tfs.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 15:20:16 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, both look doable, but Peter is the authority here... what is the timeframe for lite2/current merge? will there be a code freeze in current while it actually happens? it might work well if we could merge that exact same current with SMP at the same time, then merge SMP-current with lite2-current. where is there a lite2 tree I can examine/download? -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Dec 2 14:47:25 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA19115 for smp-outgoing; Mon, 2 Dec 1996 14:47:25 -0800 (PST) Received: from dyson.iquest.net ([198.70.144.127]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id OAA19110 for ; Mon, 2 Dec 1996 14:47:23 -0800 (PST) Received: (from root@localhost) by dyson.iquest.net (8.8.2/8.6.9) id RAA00795; Mon, 2 Dec 1996 17:47:13 -0500 (EST) From: "John S. Dyson" Message-Id: <199612022247.RAA00795@dyson.iquest.net> Subject: Re: cvs commit: sys/i386/include pmap.h To: phk@critter.tfs.com (Poul-Henning Kamp) Date: Mon, 2 Dec 1996 17:47:13 -0500 (EST) Cc: smp@csn.net, freebsd-smp@freefall.freebsd.org In-Reply-To: <3048.849564955@critter.tfs.com> from "Poul-Henning Kamp" at Dec 2, 96 11:15:55 pm X-Mailer: ELM [version 2.4 PL24 ME8] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > > smp -------------- > \ > lite2 --------- \ > \ \ > current-----------------> > > But I'm pretty sure that lite2 will get merged before smp as things > stand now. > I think that this will have to be the way that it is done. Additionally, as the pmap/vm changes for smp are merged in, I hope to be able to minimize the differences -- so that we minimize the complexity of support. Certainly, we don't want to clobber our single processor performance -- but yet we don't want to sacrifice any advantages of the SMP configurations. Darn'it I have yet to review the SMP code, but will probably do so tonight. At least it'll give me an idea of what is going on. I hope that most of the differences are in the interrupt/exception and initialization areas. Regarding Lite/2, I believe that DG, PHK or I will take an initial cut on it (the work is from Jeff Hsu.) The stuff has aged a bit (perhaps alot.) We'll all review/work on it, using each other as resources. I don't know, maybe, if I continue to feel well from my vacation, I can work on it a bit. I'll try to commit (wishy-washy, I know) to doing something by the end of the week (fat chance) -- those kinds of promises from me usually mean next week sometime... So much to do, and only 24 hours to do it in... John From owner-freebsd-smp Mon Dec 2 15:02:07 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id PAA19817 for smp-outgoing; Mon, 2 Dec 1996 15:02:07 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA19812 for ; Mon, 2 Dec 1996 15:02:05 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id SAA10867; Mon, 2 Dec 1996 18:58:22 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612022258.SAA10867@bluenose.na.tuns.ca> Subject: Re: Status report... To: erich@lodgenet.com (Eric L. Hernes) Date: Mon, 2 Dec 1996 18:58:22 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612021949.NAA04298@jake.lodgenet.com> from "Eric L. Hernes" at "Dec 2, 96 01:49:28 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > I just tested a kernel from this morning (12/1). It is *much* better > than the one I tried last week. Last week I got lots of sig11's, but > I haven't seen one yet! But I am getting a few silo overflows > from the mouse, dmesg shows: > I got the same thing. No sig11's. But when I run multiple processes, it seems that all the processes are taken care of by the same cpu and switched back and forth. On the screen of `top': RUN/#1 cpp CPU/#1 top .... RUN/#0 cpp RUN/#0 top Is the schedule supposed like that? I remember before the merge with the current, the multiple processes were RUN/#1 cpp CPU/#0 top .... Besides, if the system is rebooted, I always got the message like: I'm on cpu #1, I need to be on cpu #0! Jim From owner-freebsd-smp Mon Dec 2 16:19:33 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA23533 for smp-outgoing; Mon, 2 Dec 1996 16:19:33 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id QAA23527 for ; Mon, 2 Dec 1996 16:19:30 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id UAA11107; Mon, 2 Dec 1996 20:15:50 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612030015.UAA11107@bluenose.na.tuns.ca> Subject: Re: Status report... To: smp@csn.net (Steve Passe) Date: Mon, 2 Dec 1996 20:15:50 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612030011.RAA23839@clem.systemsix.com> from Steve Passe at "Dec 2, 96 05:11:40 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > I'm on cpu #1, I need to be on cpu #0! > > do you mean you used to get it, but now you don't? > I get this message now with the current smp-kernel. Jim From owner-freebsd-smp Mon Dec 2 16:47:19 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA24885 for smp-outgoing; Mon, 2 Dec 1996 16:47:19 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id QAA24880 for ; Mon, 2 Dec 1996 16:47:13 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id UAA11182; Mon, 2 Dec 1996 20:43:41 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612030043.UAA11182@bluenose.na.tuns.ca> Subject: Re: Status report... To: smp@csn.net (Steve Passe) Date: Mon, 2 Dec 1996 20:43:41 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612030022.RAA23908@clem.systemsix.com> from Steve Passe at "Dec 2, 96 05:22:22 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > >> > I'm on cpu #1, I need to be on cpu #0! > >> > >> do you mean you used to get it, but now you don't? > >> > >I get this message now with the current smp-kernel. > > This is normal, its just because cpu1 started to do some shutdown code > and that all has to be done by cpu0 to keep from "cutting off the limb > we're sitting on". > If it is normal, I think the sentence may have to change. Otherwise, it sounds like a `problem', especially, there is an `Oops!' in front of it! Just a thought Jim From owner-freebsd-smp Mon Dec 2 16:50:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA25006 for smp-outgoing; Mon, 2 Dec 1996 16:50:24 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id QAA24998 for ; Mon, 2 Dec 1996 16:50:19 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.3/8.8.3) with ESMTP id IAA02845; Tue, 3 Dec 1996 08:49:46 +0800 (WST) Message-Id: <199612030049.IAA02845@spinner.DIALix.COM> To: "J.M. Chuang" cc: erich@lodgenet.com (Eric L. Hernes), smp@freebsd.org Subject: Re: Status report... In-reply-to: Your message of "Mon, 02 Dec 1996 18:58:22 -0400." <199612022258.SAA10867@bluenose.na.tuns.ca> Date: Tue, 03 Dec 1996 08:49:46 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk "J.M. Chuang" wrote: > > I just tested a kernel from this morning (12/1). It is *much* better > > than the one I tried last week. Last week I got lots of sig11's, but > > I haven't seen one yet! But I am getting a few silo overflows > > from the mouse, dmesg shows: > > > I got the same thing. No sig11's. But when I run multiple processes, > it seems that all the processes are taken care of by the same cpu > and switched back and forth. On the screen of `top': [..] I know about some problems here, I bailed out last night before I fell asleep with a jumbo set of diffs to back out some more stuff again, given my .. umm... "deadly" ... track record of problems with commits while 99% asleep at the keyboard. If you were to time a 'make -j 8' or whatever, you'll currently notice that it is no faster than a 'make'.... Parallel scheduling is rather broken as a result of automatically starting the AP cpu's. I don't know why, but turning it off (again!) fixes it. > Besides, if the system is rebooted, I always got the message like: > > I'm on cpu #1, I need to be on cpu #0! Yes, according to the MPSPEC, the only cpu that is allowed to reboot or halt the system is the primary CPU. The reboot code has to try and shift itself onto cpu#0 if it starts elsewhere. > Jim Cheers, -Peter From owner-freebsd-smp Mon Dec 2 16:54:53 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA25196 for smp-outgoing; Mon, 2 Dec 1996 16:54:53 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA25189 for freebsd-smp; Mon, 2 Dec 1996 16:54:52 -0800 (PST) Date: Mon, 2 Dec 1996 16:54:52 -0800 (PST) From: Peter Wemm Message-Id: <199612030054.QAA25189@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/conf options.i386 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 16:54:51 Modified: i386/conf options.i386 Log: Add two new "about to be used" options, SERIAL_DEBUG and SMP_AUTOSTART, both expected to be off by default for the time being. Revision Changes Path 1.12 +3 -0 sys/i386/conf/options.i386 From owner-freebsd-smp Mon Dec 2 16:56:16 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA25326 for smp-outgoing; Mon, 2 Dec 1996 16:56:16 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA25316 for freebsd-smp; Mon, 2 Dec 1996 16:56:15 -0800 (PST) Date: Mon, 2 Dec 1996 16:56:15 -0800 (PST) From: Peter Wemm Message-Id: <199612030056.QAA25316@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 serial.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 16:56:15 Modified: i386/i386 serial.s Log: make this compile standalone, and use the SERIAL_DEBUG option Revision Changes Path 1.9 +21 -8 sys/i386/i386/serial.s From owner-freebsd-smp Mon Dec 2 16:57:32 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA25432 for smp-outgoing; Mon, 2 Dec 1996 16:57:32 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id QAA25423 for freebsd-smp; Mon, 2 Dec 1996 16:57:31 -0800 (PST) Date: Mon, 2 Dec 1996 16:57:31 -0800 (PST) From: Peter Wemm Message-Id: <199612030057.QAA25423@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 i386-gdbstub.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 16:57:30 Modified: i386/i386 i386-gdbstub.c Log: Quickly steal a fix from -current where this would cause link errors if there were no sio drivers in the kernel. Revision Changes Path 1.2 +9 -0 sys/i386/i386/i386-gdbstub.c From owner-freebsd-smp Mon Dec 2 17:01:27 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA25740 for smp-outgoing; Mon, 2 Dec 1996 17:01:27 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA25732 for freebsd-smp; Mon, 2 Dec 1996 17:01:26 -0800 (PST) Date: Mon, 2 Dec 1996 17:01:26 -0800 (PST) From: Peter Wemm Message-Id: <199612030101.RAA25732@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/conf options.i386 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:01:26 Modified: i386/conf options.i386 Log: recognise SMP_INVLTLB option for intercepting the invltlb*() calls and propagating them to other cpu's. This is experimental and off by default. Revision Changes Path 1.13 +1 -0 sys/i386/conf/options.i386 From owner-freebsd-smp Mon Dec 2 17:03:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA25874 for smp-outgoing; Mon, 2 Dec 1996 17:03:24 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA25867 for freebsd-smp; Mon, 2 Dec 1996 17:03:23 -0800 (PST) Date: Mon, 2 Dec 1996 17:03:23 -0800 (PST) From: Peter Wemm Message-Id: <199612030103.RAA25867@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/conf files.i386 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:03:22 Modified: i386/conf files.i386 Log: add i386/i386/serial.s as a standard file - it's got an #ifdef inside it, a bug in config(8) prevents it being "optional serial_debug" as I'd like. Revision Changes Path 1.12 +1 -0 sys/i386/conf/files.i386 From owner-freebsd-smp Mon Dec 2 17:06:02 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA25994 for smp-outgoing; Mon, 2 Dec 1996 17:06:02 -0800 (PST) Received: from root.com (implode.root.com [198.145.90.17]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA25985 for ; Mon, 2 Dec 1996 17:05:58 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by root.com (8.7.6/8.6.5) with SMTP id RAA00440; Mon, 2 Dec 1996 17:05:38 -0800 (PST) Message-Id: <199612030105.RAA00440@root.com> X-Authentication-Warning: implode.root.com: Host localhost [127.0.0.1] didn't use HELO protocol To: Peter Wemm cc: "J.M. Chuang" , erich@lodgenet.com (Eric L. Hernes), smp@freebsd.org Subject: Re: Status report... In-reply-to: Your message of "Tue, 03 Dec 1996 08:49:46 +0800." <199612030049.IAA02845@spinner.DIALix.COM> From: David Greenman Reply-To: dg@Root.COM Date: Mon, 02 Dec 1996 17:05:38 -0800 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >> Besides, if the system is rebooted, I always got the message like: >> >> I'm on cpu #1, I need to be on cpu #0! > >Yes, according to the MPSPEC, the only cpu that is allowed to reboot or >halt the system is the primary CPU. The reboot code has to try and >shift itself onto cpu#0 if it starts elsewhere. If one doesn't already exist, a mechanism needs to be created for shutting down individual CPUs. A shutdown hook should then be added that shuts down all of the CPUs except 0. -DG David Greenman Core-team/Principal Architect, The FreeBSD Project From owner-freebsd-smp Mon Dec 2 17:09:30 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26234 for smp-outgoing; Mon, 2 Dec 1996 17:09:30 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26227 for freebsd-smp; Mon, 2 Dec 1996 17:09:29 -0800 (PST) Date: Mon, 2 Dec 1996 17:09:29 -0800 (PST) From: Peter Wemm Message-Id: <199612030109.RAA26227@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/kern init_main.c init_smp.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:09:28 Modified: kern init_main.c init_smp.c Log: Split most of the SMP code out of init_main.c and into a seperate init_smp.c file (optional smp). Changes in the process: - smp sysctl stuff cleaned up and collected together - variables cleaned up - SMP_AUTOSTART code #ifdef'ed Note that I copied the cvs file underneath so that we have cvs history on it. Revision Changes Path 1.40 +2 -229 sys/kern/init_main.c 1.40 +33 -635 sys/kern/init_smp.c From owner-freebsd-smp Mon Dec 2 17:13:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26442 for smp-outgoing; Mon, 2 Dec 1996 17:13:01 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26431 for freebsd-smp; Mon, 2 Dec 1996 17:12:57 -0800 (PST) Date: Mon, 2 Dec 1996 17:12:57 -0800 (PST) From: Peter Wemm Message-Id: <199612030112.RAA26431@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/include spl.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:12:56 Modified: i386/include spl.h Log: slight cosmetic cleanup to SETBITS macro Revision Changes Path 1.6 +2 -3 sys/i386/include/spl.h From owner-freebsd-smp Mon Dec 2 17:15:07 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26561 for smp-outgoing; Mon, 2 Dec 1996 17:15:07 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26548 for freebsd-smp; Mon, 2 Dec 1996 17:15:04 -0800 (PST) Date: Mon, 2 Dec 1996 17:15:04 -0800 (PST) From: Peter Wemm Message-Id: <199612030115.RAA26548@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 locore.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:15:04 Modified: i386/i386 locore.s Log: revive phk's original "I am alive" message on the serial port when in SERIAL_DEBUG mode, this is to initialise the serial hardware and confirm that the cable works etc. Revision Changes Path 1.33 +10 -0 sys/i386/i386/locore.s From owner-freebsd-smp Mon Dec 2 17:15:02 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26538 for smp-outgoing; Mon, 2 Dec 1996 17:15:02 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA26516 for ; Mon, 2 Dec 1996 17:14:57 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id VAA11322; Mon, 2 Dec 1996 21:11:14 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612030111.VAA11322@bluenose.na.tuns.ca> Subject: Re: Status report... To: peter@spinner.dialix.com (Peter Wemm) Date: Mon, 2 Dec 1996 21:11:13 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612030049.IAA02845@spinner.DIALix.COM> from Peter Wemm at "Dec 3, 96 08:49:46 am" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > If you were to time a 'make -j 8' or whatever, you'll currently notice > that it is no faster than a 'make'.... Parallel scheduling is rather > broken as a result of automatically starting the AP cpu's. I don't know > why, but turning it off (again!) fixes it. Could the disappearance of sig11 be related to the broken parallel scheduling? Before the merge of the current, if there is only one active cpu with smp-kernel, sig11 does not show up often. Jim From owner-freebsd-smp Mon Dec 2 17:18:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26782 for smp-outgoing; Mon, 2 Dec 1996 17:18:24 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA26775 for freebsd-smp; Mon, 2 Dec 1996 17:18:23 -0800 (PST) Date: Mon, 2 Dec 1996 17:18:23 -0800 (PST) From: Peter Wemm Message-Id: <199612030118.RAA26775@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mpboot.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:18:23 Modified: i386/i386 mpboot.s Log: Second part of making SMP_AUTOSTART optional. With this option not specified (it's off by default), "sysctl -w kern.smp_active=2" is needed again. Sigh. Still, at least it works faster this way. (best I've got from 'time make -jN' is 192% usage on a 2-cpu system.) Revision Changes Path 1.11 +6 -0 sys/i386/i386/mpboot.s From owner-freebsd-smp Mon Dec 2 17:23:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA27068 for smp-outgoing; Mon, 2 Dec 1996 17:23:46 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA27061 for freebsd-smp; Mon, 2 Dec 1996 17:23:44 -0800 (PST) Date: Mon, 2 Dec 1996 17:23:44 -0800 (PST) From: Peter Wemm Message-Id: <199612030123.RAA27061@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 pmap.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:23:44 Modified: i386/i386 pmap.c Log: Add optional SMP_INVLTLB code. Note that this appears to work, but I think it's not going to be the final form. I suspect that only a minority invltlb() and invltlb_1pg()/2pg() calls need to be propagated to all cpu's. (eg: only kernel mapping changes and changes for processes actually running on another cpu). Also, watching this on the serial trace shows that we do a hell of a lot of invltlb's... (peaking at several hundred per second from what I've seen, but this is a subjective estimate and not actually measured) Revision Changes Path 1.30 +34 -24 sys/i386/i386/pmap.c From owner-freebsd-smp Mon Dec 2 17:27:25 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA27227 for smp-outgoing; Mon, 2 Dec 1996 17:27:25 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA27216 for ; Mon, 2 Dec 1996 17:27:22 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id CAA03460; Tue, 3 Dec 1996 02:27:12 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id CAA17481; Tue, 3 Dec 1996 02:27:11 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id CAA26717; Tue, 3 Dec 1996 02:27:10 +0100 (MET) Message-Id: <199612030127.CAA26717@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Hogan Whittall cc: freebsd-smp@freebsd.org Subject: Re: Still system slowdowns with certain apps In-reply-to: Your message of "Sun, 01 Dec 1996 22:08:03 MET." <199612020508.WAA02550@mystic.sedona.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Dec 1996 02:27:10 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >The new kernel works great and I've yet to have a program seg fault on >me...but ever since I got the latest source on Thursday night I've >experienced a major system slowdown whenever I compile something. The >compile itself moves right along and gets done fast, but everything else >on the system takes forever to respond until the compile is done. Before >Thursday everything worked fine whencompiling and there were still seg >faults...but this slowdown is new and I was hoping code I got today would >fix it...but it's still there. Has anyone else experienced this or know >of a fix? > Well, I have experienced the same slowdown. And it seems like things are a lot worse than just a slowdown. Thought I should run a couple of parallell dhrystones just for fun before I went home to my bed. Never got that far. The system locked up the moment I started the first and didn't allow me to do anything else until it had finished running. Then I just wrote: main() { while(1); } It didn't start so well, but just when I typed ctrl-c, it froze the system. Reset button only way out. Tried again, seems like ctrl-c doesn't have anything to do with it at least. But the second time I ran the program, things froze quite quickly. Reset button next.... Seems like it's not as effective as dhrystone (well, that loop by it self is probably not a very efficient cpu consumer, should be easy enough to improve it), but when it locks, it locks well ;) Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Mon Dec 2 17:32:37 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA27532 for smp-outgoing; Mon, 2 Dec 1996 17:32:37 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA27524 for freebsd-smp; Mon, 2 Dec 1996 17:32:36 -0800 (PST) Date: Mon, 2 Dec 1996 17:32:36 -0800 (PST) From: Peter Wemm Message-Id: <199612030132.RAA27524@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mp_machdep.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:32:35 Modified: i386/i386 mp_machdep.c Log: Add initial code for SMP_INVLTLB backend support, not enabled by default. Revision Changes Path 1.25 +44 -2 sys/i386/i386/mp_machdep.c From owner-freebsd-smp Mon Dec 2 17:34:10 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA27656 for smp-outgoing; Mon, 2 Dec 1996 17:34:10 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA27634 for freebsd-smp; Mon, 2 Dec 1996 17:34:04 -0800 (PST) Date: Mon, 2 Dec 1996 17:34:04 -0800 (PST) From: Peter Wemm Message-Id: <199612030134.RAA27634@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/include cpufunc.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:34:03 Modified: i386/include cpufunc.h Log: Add #ifdef SMP_INVLTLB hook in invltlb() Revision Changes Path 1.12 +4 -0 sys/i386/include/cpufunc.h From owner-freebsd-smp Mon Dec 2 17:46:17 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA28597 for smp-outgoing; Mon, 2 Dec 1996 17:46:17 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA28588 for freebsd-smp; Mon, 2 Dec 1996 17:46:14 -0800 (PST) Date: Mon, 2 Dec 1996 17:46:14 -0800 (PST) From: Peter Wemm Message-Id: <199612030146.RAA28588@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/include smp.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:46:14 Modified: i386/include smp.h Log: Add prototypes for pmap_bootstrap2() and smp_invltlb(). There is a problem here.. invltlb() is an inline function. smp_invltlb() is not. the IPI routines are inline. (FAST_IPI is default it seems) The reason why smp_invltlb() is not inline is because all users of cpufunc.h would also need to have the inline definitions from mpapic.h #included already, so opt_smp.h gets an even wider impact scope etc. Also, if it were inline, the size of the code generated by a call to invltlb() would be even more expensive and the cache hits would probably make it cheaper to be a plain function call with no inlining. Revision Changes Path 1.22 +5 -1 sys/i386/include/smp.h From owner-freebsd-smp Mon Dec 2 17:53:05 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA28956 for smp-outgoing; Mon, 2 Dec 1996 17:53:05 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA28947 for freebsd-smp; Mon, 2 Dec 1996 17:53:02 -0800 (PST) Date: Mon, 2 Dec 1996 17:53:02 -0800 (PST) From: Peter Wemm Message-Id: <199612030153.RAA28947@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/conf files Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 17:53:01 Modified: conf files Log: oops, nearly forgot to commit the entry for init_smp.c... (I think I've finished the commits here, I'm about to recompile since I made some "trivial" changes while committing) Revision Changes Path 1.2 +50 -35 sys/conf/files From owner-freebsd-smp Mon Dec 2 18:58:51 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA02460 for smp-outgoing; Mon, 2 Dec 1996 18:58:51 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA02450 for freebsd-smp; Mon, 2 Dec 1996 18:58:50 -0800 (PST) Date: Mon, 2 Dec 1996 18:58:50 -0800 (PST) From: Peter Wemm Message-Id: <199612030258.SAA02450@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/include spl.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 18:58:47 Modified: i386/include spl.h Log: aiee!!! Where's the pointy hat? No, it seems that I cannot assume that two sequential inline __asm __volatile statements do not have any code in between them. I kinda forgot that the compiler may generate code to support the second line. And, for some reason, "lock; movl $0x40000000,%eax" isn't allowed.. :-] Revision Changes Path 1.7 +3 -2 sys/i386/include/spl.h From owner-freebsd-smp Mon Dec 2 19:37:00 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA04519 for smp-outgoing; Mon, 2 Dec 1996 19:37:00 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id TAA04514; Mon, 2 Dec 1996 19:36:56 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id UAA25088; Mon, 2 Dec 1996 20:36:53 -0700 Message-Id: <199612030336.UAA25088@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Peter Wemm cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include spl.h In-reply-to: Your message of "Mon, 02 Dec 1996 18:58:50 PST." <199612030258.SAA02450@freefall.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 20:36:52 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, this is more like it, my best time with older working kernel was 269: % time make -j8 ... 271.22s real 414.10s user 87.29s system ^^^ ^^^^ ^^^ ^^^^ and the "jerkiness" is gone (again, now I remember we had this same symptom last time we played with 'autostart'...) and I think the missing INTs are gone (hope I didn't just jinx that!) -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Dec 2 19:53:02 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA05054 for smp-outgoing; Mon, 2 Dec 1996 19:53:02 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA05048 for ; Mon, 2 Dec 1996 19:52:53 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.3/8.8.3) with ESMTP id LAA01320; Tue, 3 Dec 1996 11:52:39 +0800 (WST) Message-Id: <199612030352.LAA01320@spinner.DIALix.COM> To: Steve Passe cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include spl.h In-reply-to: Your message of "Mon, 02 Dec 1996 20:36:52 MST." <199612030336.UAA25088@clem.systemsix.com> Date: Tue, 03 Dec 1996 11:52:39 +0800 From: Peter Wemm Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Steve Passe wrote: > Hi, > > this is more like it, my best time with older working kernel was 269: > > % time make -j8 > ... > 271.22s real 414.10s user 87.29s system > ^^^ ^^^^ ^^^ ^^^^ > > and the "jerkiness" is gone (again, now I remember we had this same symptom > last time we played with 'autostart'...) I still don't know what happened, I'd love to know what's going on. I almost wonder if both cpu's were running as an exact mirror of each other.. Anyway, this is what I hope to diagnose now that we have serial debugging again. > and I think the missing INTs are gone (hope I didn't just jinx that!) Seconded.. :-) Anyway, we're almost back to where we started from before the chaos, except this time the code is #ifdef'ed and we have serial trace code to debug it.. Cheers, -Peter From owner-freebsd-smp Mon Dec 2 20:35:00 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA06847 for smp-outgoing; Mon, 2 Dec 1996 20:35:00 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id UAA06829 for ; Mon, 2 Dec 1996 20:34:52 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id AAA11959; Tue, 3 Dec 1996 00:28:32 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612030428.AAA11959@bluenose.na.tuns.ca> Subject: Re: cvs commit: sys/i386/include spl.h To: peter@spinner.dialix.com (Peter Wemm) Date: Tue, 3 Dec 1996 00:28:31 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612030352.LAA01320@spinner.DIALix.COM> from Peter Wemm at "Dec 3, 96 11:52:39 am" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > > % time make -j8 > > ... > > 271.22s real 414.10s user 87.29s system > > ^^^ ^^^^ ^^^ ^^^^ > > > > and the "jerkiness" is gone (again, now I remember we had this same symptom > > last time we played with 'autostart'...) > > I still don't know what happened, I'd love to know what's going on. > > I almost wonder if both cpu's were running as an exact mirror of each > other.. > > Anyway, this is what I hope to diagnose now that we have serial debugging > again. > > > and I think the missing INTs are gone (hope I didn't just jinx that!) > > Seconded.. :-) > > Anyway, we're almost back to where we started from before the chaos, > except this time the code is #ifdef'ed and we have serial trace code > to debug it.. > It looks like even `silo overflow' is gone! Good Work! Jim From owner-freebsd-smp Mon Dec 2 20:42:31 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA07360 for smp-outgoing; Mon, 2 Dec 1996 20:42:31 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id UAA07353 for ; Mon, 2 Dec 1996 20:42:24 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.3/8.8.3) with ESMTP id MAA01634; Tue, 3 Dec 1996 12:41:11 +0800 (WST) Message-Id: <199612030441.MAA01634@spinner.DIALix.COM> To: "J.M. Chuang" cc: smp@freebsd.org Subject: Re: cvs commit: sys/i386/include spl.h In-reply-to: Your message of "Tue, 03 Dec 1996 00:28:31 -0400." <199612030428.AAA11959@bluenose.na.tuns.ca> Date: Tue, 03 Dec 1996 12:41:11 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk "J.M. Chuang" wrote: [..] > It looks like even `silo overflow' is gone! Good Work! Yeah, but it's a shame that in order to do this I had to disable just about everything I've worked on for the last few weeks.. :-( And we're still no closer to figuring out why this happens. :-( "options SMP_AUTOSTART" appears to be the culprit, which is pretty much necessary for >2 cpu support.. :-( Thanks for the feedback though.. It's good to know we can stop sweating about it and start trying to work out why. > Jim Cheers, -Peter From owner-freebsd-smp Mon Dec 2 20:49:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA07705 for smp-outgoing; Mon, 2 Dec 1996 20:49:46 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA07697 for freebsd-smp; Mon, 2 Dec 1996 20:49:44 -0800 (PST) Date: Mon, 2 Dec 1996 20:49:44 -0800 (PST) From: Peter Wemm Message-Id: <199612030449.UAA07697@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mpboot.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 20:49:42 Modified: i386/i386 mpboot.s Log: mpboot.s was missing the line: #include "opt_smp.h" Steve noticed that boot_unlock was undefined when he tried SMP_AUTOSTART. Revision Changes Path 1.12 +1 -0 sys/i386/i386/mpboot.s From owner-freebsd-smp Mon Dec 2 20:49:35 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA07679 for smp-outgoing; Mon, 2 Dec 1996 20:49:35 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id UAA07674 for ; Mon, 2 Dec 1996 20:49:29 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id VAA25502; Mon, 2 Dec 1996 21:49:08 -0700 Message-Id: <199612030449.VAA25502@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: "J.M. Chuang" cc: smp@FreeBSD.org Subject: Re: cvs commit: sys/i386/include spl.h In-reply-to: Your message of "Tue, 03 Dec 1996 00:28:31 -0400." <199612030428.AAA11959@bluenose.na.tuns.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 02 Dec 1996 21:49:08 -0700 Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk Hi, >> > and I think the missing INTs are gone (hope I didn't just jinx that!) >> >> Seconded.. :-) >> >> Anyway, we're almost back to where we started from before the chaos, >> except this time the code is #ifdef'ed and we have serial trace code >> to debug it.. >> >It looks like even `silo overflow' is gone! Good Work! lost INTs, silo overflows, network card timeouts are all symptoms of something screwy in the "autostart" code. This is now off by default and the SMP-current kernel is stable at this point. I am very curious to here from one of the EIDE testers to see if we fixed that problem also... Now's the time to update your cvs tree, gang, even if you don't have time for testing... -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Dec 2 20:59:54 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id UAA08284 for smp-outgoing; Mon, 2 Dec 1996 20:59:54 -0800 (PST) Received: from gargoyle.bazzle.com (gargoyle.bazzle.com [206.103.246.190]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id UAA08271 for ; Mon, 2 Dec 1996 20:59:42 -0800 (PST) Received: from gargoyle.bazzle.com (localhost [127.0.0.1]) by gargoyle.bazzle.com (8.8.3/8.6.12) with SMTP id XAA01392; Mon, 2 Dec 1996 23:59:38 -0500 (EST) Date: Mon, 2 Dec 1996 23:59:38 -0500 (EST) From: "Eric J. Chet" To: smp@freebsd.org cc: dob@nasvr1.cb.lucent.com Subject: We have a pulse Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve & SMP'ers We have a pulse :-) The first GigaByte 586DX mainboard I received was DOA, the second is working great. I just installed the latest smp-kernel without any problems. I have include the output from mptable below and my kernel config file. There is only one difference between my mptabe and yours it's the first line. Kernel compile times: smp-kernel compiled under -current (as of 12/01) >time make > 314.39s real 282.00s user 18.72s system smp-kernel compiled under -smp-current >time make -j8 > 213.57s real 338.08s user 77.12s system We are on our way. Great job everybody. Thanks, Eric J. Chet - ejc@bazzle.com =============================================================================== MPTable, version 2.0.4 looking for EBDA pointer @ 0x040e, found, searching EBDA @ 0x0009fc00 searching CMOS 'top of mem' @ 0x0009f800 (638K) searching default 'top of mem' @ 0x0009fc00 (639K) searching BIOS @ 0x000f0000 MP FPS found in BIOS @ physical addr: 0x000f0c80 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000f0c80 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0xf4 mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f0c94 signature: 'PCMP' base table length: 292 version: 1.1 checksum: 0x31 OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 28 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x11 BSP, usable 5 2 1 0x07bf 1 0x11 AP, usable 5 2 1 0x07bf -- Bus: Bus ID Type 0 ISA 1 PCI -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT conforms conforms 0 0 2 0 INT conforms conforms 0 1 2 1 INT conforms conforms 0 0 2 2 INT conforms conforms 0 3 2 3 INT conforms conforms 0 4 2 4 INT conforms conforms 0 5 2 5 INT conforms conforms 0 6 2 6 INT conforms conforms 0 7 2 7 INT conforms conforms 0 8 2 8 INT conforms conforms 0 9 2 9 INT conforms conforms 0 10 2 10 INT conforms conforms 0 11 2 11 INT conforms conforms 0 12 2 12 INT conforms conforms 0 13 2 13 INT conforms conforms 0 14 2 14 INT conforms conforms 0 15 2 15 INT active-lo level 1 8:A 2 16 INT active-lo level 1 9:A 2 17 INT active-lo level 1 10:A 2 18 INT active-lo level 1 12:A 2 19 SMI conforms conforms 0 0 2 23 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT active-hi edge 0 0 255 0 NMI active-hi edge 0 0 255 1 ------------------------------------------------------------------------------- dmesg output: init386 done CR0 = 80000011 Copyright (c) 1992-1996 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.0-SMP #0: Mon Dec 2 23:17:41 EST 1996 ejc@gargoyle.bazzle.com:/var/usr/src/sys-smp/compile/gargoyle-smp FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010 cpu1 (AP): apic id: 1, version: 0x00030010 io0 (APIC): apic id: 2, version: 0x00170011 Calibrating clock(s) relative to mc146818A clock ... i8254 clock: 1193333 Hz CPU: Pentium (586-class CPU) Origin = "GenuineIntel" Id = 0x526 Stepping=6 Features=0x3bf real memory = 67108864 (65536K bytes) avail memory = 63721472 (62228K bytes) Probing for devices on PCI bus 0: chip0 rev 1 on pci0:0 chip1 rev 1 on pci0:7:0 chip2 rev 0 on pci0:7:1 vga0 rev 1 on pci0:8 ahc0 rev 0 int a irq 19 on pci0:12 Freeing (NOT implimented) irq 11 for ISA cards. ahc0: aic7880 Wide Channel, SCSI Id=7, 16/255 SCBs ahc0 waiting for scsi devices to settle ahc0: target 8 Tagged Queuing Device (ahc0:8:0): "SEAGATE ST15150W 0020" type 0 fixed SCSI 2 sd0(ahc0:8:0): Direct-Access 4095MB (8388315 512 byte sectors) sd0(ahc0:8:0): with 3712 cyls, 21 heads, and an average 107 sectors/track Probing for devices on the ISA bus: sc0 at 0x60-0x6f irq 1 on motherboard sc0: VGA color <16 virtual consoles, flags=0x0> sio0 at 0x3f8-0x3ff irq 4 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A psm0 at 0x60-0x64 irq 12 on motherboard psm0: device ID 0, 3 buttons? lpt0 at 0x378-0x37f irq 7 on isa lpt0: Interrupt-driven port lp0: TCP/IP capable interface fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: NEC 72065B fd0: 1.44MB 3.5in npx0 on motherboard npx0: INT 16 interface Enabled INTs: 1, 2, 3, 4, 6, 7, 8, 12, 19, imen: 0x00f7ee21 ------------------------------------------------------------------------------- # SMP kernel config file options: options SMP # Symmetric MultiProcessor Kernel #options APIC_IO # Symmetric (APIC) I/O options NCPU=2 # number of CPUs options NBUS=2 # number of busses options NAPIC=1 # number of IO APICs options NINTR=21 # number of INTs =============================================================================== machine "i386" cpu "I586_CPU" ident gargoyle maxusers 32 options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O options NCPU=2 # number of CPUs options NBUS=2 # number of busses options NAPIC=1 # number of IO APICs options NINTR=21 # number of INTs options INET #InterNETworking options FFS #Berkeley Fast Filesystem options MFS #Memory Filesystem options MSDOSFS #MSDOS Filesystem options "CD9660" #ISO 9660 Filesystem options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15 #Be pessimistic about Joe SCSI device options UCONSOLE #Allow users to grab the console #options BROKEN_KEYBOARD_RESET options QUOTA #options PQ_MEDIUMCACHE #256k of L2 cache options SYSVSHM options SYSVSEM options SYSVMSG options CHILD_MAX=128 options OPEN_MAX=128 options VISUAL_USERCONFIG options USERCONFIG options DDB options DDB_UNATTENDED #options NFS #Network Filesystem #options IPFIREWALL #options IPFIREWALL_VERBOSE #options NMBCLUSTERS=2048 options PQ_LARGECACHE #512k of L2 cache options "SHMMAXPGS=1280" options KTRACE #options USER_LDT # # PCI devices: # # The main PCI bus device is `pci'. It provides auto-detection and # configuration support for all devices on the PCI bus, using either # configuration mode defined in the PCI specification. # # The `ahc' device provides support for the Adaptec 29/3940(U)(W) # and motherboard based AIC7870/AIC7880 adapters. # # enable tagged command queueing, which is a major performance win on # devices that support it (and controllers with enough SCB's) options AHC_TAGENABLE # enable SCB paging - See the ahc.4 man page options AHC_SCBPAGING_ENABLE # The aic7xxx driver will attempt to use memory mapped I/O for all PCI # controllers that have it configured only if this option is set. Unfortunately, # this doesn't work on some motherboards, which prevents it from being the # default. options AHC_ALLOW_MEMIO # SCSI OPTIONS: # SCSIDEBUG: When defined enables debugging macros # NO_SCSI_SENSE: When defined disables sense descriptions (about 4k) # SCSI_REPORT_GEOMETRY: Always report disk geometry at boot up instead # of only when booting verbosely. #options SCSIDEBUG #options NO_SCSI_SENSE options SCSI_REPORT_GEOMETRY config kernel root on sd0 controller isa0 #controller eisa0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 vector fdintr disk fd0 at fdc0 drive 0 disk fd1 at fdc0 drive 1 #tape ft0 at fdc0 drive 2 controller pci0 controller ahc0 #controller ncr0 #controller scbus0 at ncr0 controller scbus0 at ahc0 disk sd0 at scbus0 target 8 disk sd1 at scbus0 target 1 device cd0 at scbus0 target 2 #device sd0 #device cd0 #Only need one of these, the code dynamically grows # syscons is the default console driver, resembling an SCO console device sc0 at isa? port "IO_KBD" tty irq 1 vector scintr # Enable this and PCVT_FREEBSD for pcvt vt220 compatible console driver #device vt0 at isa? port "IO_KBD" tty irq 1 vector pcrint #options PCVT_FREEBSD=210 # pcvt running on FreeBSD >= 2.0.5 options XSERVER # include code for XFree86 #options FAT_CURSOR # start with block cursor # If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines #options PCVT_SCANSET=2 # IBM keyboards are non-std # Mandatory, don't remove device npx0 at isa? port "IO_NPX" irq 13 vector npxintr device sio0 at isa? port "IO_COM1" tty irq 4 vector siointr device sio1 at isa? port "IO_COM2" tty irq 3 vector siointr #device sio2 at isa? port "IO_COM3" tty irq 5 vector siointr #device sio3 at isa? disable port "IO_COM4" tty irq 9 vector siointr device psm0 at isa? port "IO_KBD" conflicts tty irq 12 vector psmintr device lpt0 at isa? port? tty irq 7 vector lptintr pseudo-device loop pseudo-device ether pseudo-device log #pseudo-device sl 1 # ijppp uses tun instead of ppp device pseudo-device ppp 2 pseudo-device tun 2 pseudo-device pty 16 # keep this if you want to be able to continue to use /stand/sysinstall pseudo-device gzip # Exec gzipped a.out's pseudo-device bpfilter 2 pseudo-device vn #Vnode driver (turns a file into a device) pseudo-device snp 2 #Snoop device - to look at pty/vty/etc.. From owner-freebsd-smp Mon Dec 2 21:05:07 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id VAA08673 for smp-outgoing; Mon, 2 Dec 1996 21:05:07 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id UAA08268 for ; Mon, 2 Dec 1996 20:59:38 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id AAA12107; Tue, 3 Dec 1996 00:55:18 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612030455.AAA12107@bluenose.na.tuns.ca> Subject: Re: cvs commit: sys/i386/include spl.h To: smp@csn.net (Steve Passe) Date: Tue, 3 Dec 1996 00:55:17 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612030449.VAA25502@clem.systemsix.com> from Steve Passe at "Dec 2, 96 09:49:08 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > screwy in the "autostart" code. This is now off by default and the SMP-current > kernel is stable at this point. I am very curious to here from one of the EIDE > testers to see if we fixed that problem also... > I tested. The system can boot without any problem but if the second cpu is launched, the coredump occurrs to almost every binary. If I boot the system with the SCSI and fsck IDE, there are lots of DUP's.... The filesystem is screwed up (I believe). Jim From owner-freebsd-smp Mon Dec 2 21:51:19 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id VAA10778 for smp-outgoing; Mon, 2 Dec 1996 21:51:19 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id VAA10769 for freebsd-smp; Mon, 2 Dec 1996 21:51:17 -0800 (PST) Date: Mon, 2 Dec 1996 21:51:17 -0800 (PST) From: Peter Wemm Message-Id: <199612030551.VAA10769@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 locore.s mp_machdep.c mpboot.s pmap.c swtch.s sys/kern init_smp.c sys/i386/conf options.i386 sys/i386/include smp.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/02 21:51:17 Modified: i386/conf options.i386 i386/i386 locore.s mp_machdep.c mpboot.s pmap.c swtch.s i386/include smp.h kern init_smp.c Log: Move experimental options from opt_smp.h to their own file.. Otherwise, too much needs to be recompiled when enabling one of them for testing.. Revision Changes Path 1.14 +5 -3 sys/i386/conf/options.i386 1.34 +1 -0 sys/i386/i386/locore.s 1.26 +2 -1 sys/i386/i386/mp_machdep.c 1.13 +1 -1 sys/i386/i386/mpboot.s 1.31 +1 -0 sys/i386/i386/pmap.c 1.29 +1 -0 sys/i386/i386/swtch.s 1.23 +2 -1 sys/i386/include/smp.h 1.41 +2 -1 sys/kern/init_smp.c From owner-freebsd-smp Mon Dec 2 22:02:10 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id WAA11341 for smp-outgoing; Mon, 2 Dec 1996 22:02:10 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id WAA11333 for ; Mon, 2 Dec 1996 22:02:08 -0800 (PST) Received: from critter.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0vUnvA-0003wjC; Mon, 2 Dec 96 22:01 PST Received: from critter.tfs.com (localhost.phk.dk [127.0.0.1]) by critter.tfs.com (8.8.2/8.8.2) with ESMTP id HAA03550; Tue, 3 Dec 1996 07:02:45 +0100 (MET) To: Steve Passe cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-reply-to: Your message of "Mon, 02 Dec 1996 15:20:16 MST." <199612022220.PAA23260@clem.systemsix.com> Date: Tue, 03 Dec 1996 07:02:45 +0100 Message-ID: <3548.849592965@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk In message <199612022220.PAA23260@clem.systemsix.com>, Steve Passe writes: >Hi, > >both look doable, but Peter is the authority here... > >what is the timeframe for lite2/current merge? For all I care we can do it next week when 2.2-BETA is out of the door. And I think we should ;-) >will there be a code freeze in current while it actually happens? no, it will merely be broken for a day or two. >where is there a lite2 tree I can examine/download? /home/lite2 on freefall. Sure, there's even a cvsup target for it. -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Mon Dec 2 22:54:44 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id WAA15175 for smp-outgoing; Mon, 2 Dec 1996 22:54:44 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id WAA15170 for ; Mon, 2 Dec 1996 22:54:42 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id AAA12221 for ; Tue, 3 Dec 1996 00:54:41 -0600 (CST) Message-Id: <199612030654.AAA12221@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: freebsd-smp@freebsd.org Subject: spin locks... Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Dec 1996 00:54:40 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Its something we could use, so I thought I'd give it a shot. I thought it would make a nice first assembly project, right? Yeah.. sure. I have never been so frustrated with something so obviously simple. When I try to link it into a test program, it segfaults. The only way I could obtain predictable behavior was to take out the cmpxchg and lock. Could someone please point out my error here? .text .align 2,0x90 .globl _get_spinlock _get_spinlock: pushl %eax pushl %ecx pushl %edx movl _smp_active, %eax cmpl $0, %eax jne 1f jmp 4 1: movl 4(%esp), %edx movl $1, %ecx 2: lock cmpxchg %ecx, (%edx) je 3f jmp 4 3: cmpl %ecx, (%edx) je 3b jmp 2b 4: popl %edx popl %ecx popl %eax ret --Chris Csanady From owner-freebsd-smp Mon Dec 2 23:21:51 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id XAA16073 for smp-outgoing; Mon, 2 Dec 1996 23:21:51 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id XAA16068 for ; Mon, 2 Dec 1996 23:21:47 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.3/8.8.3) with ESMTP id PAA01103; Tue, 3 Dec 1996 15:21:33 +0800 (WST) Message-Id: <199612030721.PAA01103@spinner.DIALix.COM> To: Chris Csanady cc: freebsd-smp@freebsd.org Subject: Re: spin locks... In-reply-to: Your message of "Tue, 03 Dec 1996 00:54:40 CST." <199612030654.AAA12221@friley216.res.iastate.edu> Date: Tue, 03 Dec 1996 15:21:32 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Chris Csanady wrote: > pushl %eax > pushl %ecx > pushl %edx > movl _smp_active, %eax > cmpl $0, %eax > jne 1f > jmp 4 > 1: movl 4(%esp), %edx > movl $1, %ecx > 2: lock > cmpxchg %ecx, (%edx) > je 3f > jmp 4 > 3: cmpl %ecx, (%edx) [ ... ] You've been bitten by Intel's assembler syntax.... cmpxchg actually has three arguments, %eax is implicit. The way it really works.. Given: lock; cmpxchg %ecx, (%edx), what happens is that if %eax == (%edx), it exchanges (%edx) with %ecx. ie: %eax is what you hope that it is currently set to, and %ecx is the new value if you don't loose a race. -Peter From owner-freebsd-smp Mon Dec 2 23:40:57 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id XAA16889 for smp-outgoing; Mon, 2 Dec 1996 23:40:57 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id XAA16880 for ; Mon, 2 Dec 1996 23:40:55 -0800 (PST) Received: from critter.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0vUpSo-0003vuC; Mon, 2 Dec 96 23:40 PST Received: from critter.tfs.com (localhost.phk.dk [127.0.0.1]) by critter.tfs.com (8.8.2/8.8.2) with ESMTP id IAA03917; Tue, 3 Dec 1996 08:42:04 +0100 (MET) To: Chris Csanady cc: freebsd-smp@freebsd.org Subject: Re: spin locks... In-reply-to: Your message of "Tue, 03 Dec 1996 00:54:40 CST." <199612030654.AAA12221@friley216.res.iastate.edu> Date: Tue, 03 Dec 1996 08:42:04 +0100 Message-ID: <3915.849598924@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk In message <199612030654.AAA12221@friley216.res.iastate.edu>, Chris Csanady wri tes: >Its something we could use, so I thought I'd give it a shot. I thought it >would make a nice first assembly project, right? Yeah.. sure. I have never >been so frustrated with something so obviously simple. > >When I try to link it into a test program, it segfaults. The only way I >could obtain predictable behavior was to take out the cmpxchg and lock. > >Could someone please point out my error here? > >.text > .align 2,0x90 >.globl _get_spinlock >_get_spinlock: > pushl %eax > pushl %ecx > pushl %edx > movl _smp_active, %eax > cmpl $0, %eax + je 4 -> jne 1f -> jmp 4 >1: movl 4(%esp), %edx > movl $1, %ecx >2: lock > cmpxchg %ecx, (%edx) I can't remember, that lock may be implied in the cmpxchg instruction... Check out the smplock() I wrote and see what I do. -> je 3f -> jmp 4 + jne 4 >3: cmpl %ecx, (%edx) + jne 2b > je 3b I'm sure not, but where do you want to go then ? -> jmp 2b >4: popl %edx > popl %ecx > popl %eax > ret > >--Chris Csanady > > > -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Tue Dec 3 03:42:19 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA04426 for smp-outgoing; Tue, 3 Dec 1996 03:42:19 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA04418 for freebsd-smp; Tue, 3 Dec 1996 03:42:16 -0800 (PST) Date: Tue, 3 Dec 1996 03:42:16 -0800 (PST) From: Peter Wemm Message-Id: <199612031142.DAA04418@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys - Imported sources Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 03:42:14 sys - Imported sources Update of /home/smp/sys In directory freefall.freebsd.org:/f/peter/work/sys Revision/Branch: 1.1.1 Log Message: Import -current 961203 Status: Vendor Tag: CURRENT Release Tags: v961203 I sys/CVS U sys/Makefile I sys/compile/CVS U sys/compile/.keep_me I sys/netkey/CVS U sys/netkey/key.c U sys/netkey/key.h U sys/netkey/key_debug.c U sys/netkey/key_debug.h I sys/conf/CVS U sys/conf/defines U sys/conf/files U sys/conf/files.newconf C sys/conf/newvers.sh U sys/conf/nfsswapkernel.c U sys/conf/options U sys/conf/param.c U sys/conf/systags.sh I sys/ddb/CVS U sys/ddb/db_access.c U sys/ddb/db_access.h U sys/ddb/db_aout.c U sys/ddb/db_break.c U sys/ddb/db_break.h U sys/ddb/db_command.c U sys/ddb/db_command.h U sys/ddb/db_examine.c U sys/ddb/db_expr.c U sys/ddb/db_input.c U sys/ddb/db_lex.c U sys/ddb/db_lex.h U sys/ddb/db_output.c U sys/ddb/db_output.h U sys/ddb/db_print.c U sys/ddb/db_ps.c U sys/ddb/db_run.c U sys/ddb/db_sym.c U sys/ddb/db_sym.h U sys/ddb/db_trap.c U sys/ddb/db_variables.c U sys/ddb/db_variables.h U sys/ddb/db_watch.c U sys/ddb/db_watch.h U sys/ddb/db_write_cmd.c U sys/ddb/ddb.h I sys/dev/CVS I sys/dev/aic7xxx/CVS U sys/dev/aic7xxx/aic7xxx.seq U sys/dev/aic7xxx/aic7xxx_asm.1 U sys/dev/aic7xxx/aic7xxx_asm.c U sys/dev/aic7xxx/aic7xxx_reg.h I sys/dev/vn/CVS U sys/dev/vn/vn.c I sys/dev/ccd/CVS U sys/dev/ccd/ccd.c I sys/dev/advansys/CVS U sys/dev/advansys/advlib.c U sys/dev/advansys/advlib.h U sys/dev/advansys/advmcode.c U sys/dev/advansys/advmcode.h I sys/dev/vx/CVS U sys/dev/vx/if_vx.c U sys/dev/vx/if_vxreg.h I sys/gnu/CVS I sys/gnu/ext2fs/CVS U sys/gnu/ext2fs/COPYRIGHT.INFO U sys/gnu/ext2fs/ext2_alloc.c U sys/gnu/ext2fs/ext2_balloc.c U sys/gnu/ext2fs/ext2_extern.h U sys/gnu/ext2fs/ext2_fs.h U sys/gnu/ext2fs/ext2_fs_i.h U sys/gnu/ext2fs/ext2_fs_sb.h U sys/gnu/ext2fs/ext2_inode.c U sys/gnu/ext2fs/ext2_inode_cnv.c U sys/gnu/ext2fs/ext2_linux_balloc.c U sys/gnu/ext2fs/ext2_linux_ialloc.c U sys/gnu/ext2fs/ext2_lookup.c U sys/gnu/ext2fs/ext2_readwrite.c U sys/gnu/ext2fs/ext2_subr.c U sys/gnu/ext2fs/ext2_vfsops.c U sys/gnu/ext2fs/ext2_vnops.c U sys/gnu/ext2fs/fs.h U sys/gnu/ext2fs/i386-bitops.h I sys/gnu/i386/CVS I sys/gnu/i386/fpemul/CVS U sys/gnu/i386/fpemul/Changelog U sys/gnu/i386/fpemul/README U sys/gnu/i386/fpemul/control_w.h U sys/gnu/i386/fpemul/div_small.s U sys/gnu/i386/fpemul/errors.c U sys/gnu/i386/fpemul/exception.h U sys/gnu/i386/fpemul/fpu_arith.c U sys/gnu/i386/fpemul/fpu_asm.h U sys/gnu/i386/fpemul/fpu_aux.c U sys/gnu/i386/fpemul/fpu_emu.h U sys/gnu/i386/fpemul/fpu_entry.c U sys/gnu/i386/fpemul/fpu_etc.c U sys/gnu/i386/fpemul/fpu_proto.h U sys/gnu/i386/fpemul/fpu_system.h U sys/gnu/i386/fpemul/fpu_trig.c U sys/gnu/i386/fpemul/get_address.c U sys/gnu/i386/fpemul/load_store.c U sys/gnu/i386/fpemul/math_emu.h U sys/gnu/i386/fpemul/poly_2xm1.c U sys/gnu/i386/fpemul/poly_atan.c U sys/gnu/i386/fpemul/poly_div.s U sys/gnu/i386/fpemul/poly_l2.c U sys/gnu/i386/fpemul/poly_mul64.s U sys/gnu/i386/fpemul/poly_sin.c U sys/gnu/i386/fpemul/poly_tan.c U sys/gnu/i386/fpemul/polynomial.s U sys/gnu/i386/fpemul/reg_add_sub.c U sys/gnu/i386/fpemul/reg_compare.c U sys/gnu/i386/fpemul/reg_constant.c U sys/gnu/i386/fpemul/reg_constant.h U sys/gnu/i386/fpemul/reg_div.s U sys/gnu/i386/fpemul/reg_ld_str.c U sys/gnu/i386/fpemul/reg_mul.c U sys/gnu/i386/fpemul/reg_norm.s U sys/gnu/i386/fpemul/reg_round.s U sys/gnu/i386/fpemul/reg_u_add.s U sys/gnu/i386/fpemul/reg_u_div.s U sys/gnu/i386/fpemul/reg_u_mul.s U sys/gnu/i386/fpemul/reg_u_sub.s U sys/gnu/i386/fpemul/status_w.h U sys/gnu/i386/fpemul/version.h U sys/gnu/i386/fpemul/wm_shrx.s U sys/gnu/i386/fpemul/wm_sqrt.s I sys/gnu/i386/isa/CVS U sys/gnu/i386/isa/dgb.c U sys/gnu/i386/isa/dgbios.h U sys/gnu/i386/isa/dgfep.h U sys/gnu/i386/isa/dgreg.h I sys/i386/CVS U sys/i386/Makefile I sys/i386/apm/CVS U sys/i386/apm/apm.c U sys/i386/apm/apm_setup.h U sys/i386/apm/apm_setup.s I sys/i386/apm/apm_init/CVS U sys/i386/apm/apm_init/Makefile U sys/i386/apm/apm_init/apm_init.S U sys/i386/apm/apm_init/apm_init.inc U sys/i386/apm/apm_init/bin2asm.c U sys/i386/apm/apm_init/real_prot.S U sys/i386/apm/apm_init/real_prot.h U sys/i386/apm/apm_init/rmaouthdr U sys/i386/apm/apm_init/table.c I sys/i386/boot/CVS U sys/i386/boot/Makefile I sys/i386/boot/biosboot/CVS U sys/i386/boot/biosboot/Makefile U sys/i386/boot/biosboot/README.386BSD U sys/i386/boot/biosboot/README.MACH U sys/i386/boot/biosboot/README.serial U sys/i386/boot/biosboot/asm.S U sys/i386/boot/biosboot/asm.h U sys/i386/boot/biosboot/bios.S U sys/i386/boot/biosboot/boot.c U sys/i386/boot/biosboot/boot.h U sys/i386/boot/biosboot/boot2.S U sys/i386/boot/biosboot/disk.c U sys/i386/boot/biosboot/io.c U sys/i386/boot/biosboot/probe_keyboard.c U sys/i386/boot/biosboot/serial.S U sys/i386/boot/biosboot/start.S U sys/i386/boot/biosboot/sys.c U sys/i386/boot/biosboot/table.c I sys/i386/boot/rawboot/CVS U sys/i386/boot/rawboot/Makefile U sys/i386/boot/rawboot/README I sys/i386/boot/dosboot/CVS U sys/i386/boot/dosboot/Makefile U sys/i386/boot/dosboot/ansi.h U sys/i386/boot/dosboot/boot.c U sys/i386/boot/dosboot/boot.h U sys/i386/boot/dosboot/bootinfo.h U sys/i386/boot/dosboot/cdefs.h U sys/i386/boot/dosboot/dinode.h U sys/i386/boot/dosboot/dir.h U sys/i386/boot/dosboot/dirent.h U sys/i386/boot/dosboot/disk.c U sys/i386/boot/dosboot/disklabe.h U sys/i386/boot/dosboot/dkbad.h U sys/i386/boot/dosboot/dosboot.c U sys/i386/boot/dosboot/dosboot.h U sys/i386/boot/dosboot/endian.h U sys/i386/boot/dosboot/exec.h U sys/i386/boot/dosboot/fbsdboot.c U sys/i386/boot/dosboot/fbsdboot.exe.uu U sys/i386/boot/dosboot/fbsdboot.mak U sys/i386/boot/dosboot/fs.h U sys/i386/boot/dosboot/imgact.h U sys/i386/boot/dosboot/inode.h U sys/i386/boot/dosboot/mexec.h U sys/i386/boot/dosboot/param.h U sys/i386/boot/dosboot/quota.h U sys/i386/boot/dosboot/protmod.c U sys/i386/boot/dosboot/protmod.h U sys/i386/boot/dosboot/readme U sys/i386/boot/dosboot/reboot.h U sys/i386/boot/dosboot/sys.c U sys/i386/boot/dosboot/syslimit.h U sys/i386/boot/dosboot/sysparam.h U sys/i386/boot/dosboot/types.h I sys/i386/boot/kzipboot/CVS U sys/i386/boot/kzipboot/Makefile U sys/i386/boot/kzipboot/README U sys/i386/boot/kzipboot/boot.c U sys/i386/boot/kzipboot/gzip.h U sys/i386/boot/kzipboot/head.S U sys/i386/boot/kzipboot/malloc.c U sys/i386/boot/kzipboot/misc.c U sys/i386/boot/kzipboot/tail.S U sys/i386/boot/kzipboot/unzip.c I sys/i386/boot/netboot/CVS U sys/i386/boot/netboot/3c509.c U sys/i386/boot/netboot/3c509.h U sys/i386/boot/netboot/Makefile U sys/i386/boot/netboot/bootmenu.c U sys/i386/boot/netboot/main.c U sys/i386/boot/netboot/makerom.c U sys/i386/boot/netboot/misc.c U sys/i386/boot/netboot/netboot.h U sys/i386/boot/netboot/ns8390.c U sys/i386/boot/netboot/ns8390.h U sys/i386/boot/netboot/rpc.c U sys/i386/boot/netboot/start2.S I sys/i386/conf/CVS U sys/i386/conf/GENERIC U sys/i386/conf/LINT U sys/i386/conf/Makefile.i386 U sys/i386/conf/devices.i386 C sys/i386/conf/files.i386 U sys/i386/conf/majors.i386 U sys/i386/conf/options.i386 I sys/i386/eisa/CVS U sys/i386/eisa/3c5x9.c U sys/i386/eisa/aha1742.c U sys/i386/eisa/aic7770.c U sys/i386/eisa/bt74x.c U sys/i386/eisa/eisaconf.c U sys/i386/eisa/eisaconf.h U sys/i386/eisa/if_vx_eisa.c I sys/i386/i386/CVS U sys/i386/i386/autoconf.c U sys/i386/i386/cons.c U sys/i386/i386/cons.h U sys/i386/i386/db_disasm.c U sys/i386/i386/db_interface.c U sys/i386/i386/db_trace.c U sys/i386/i386/exception.s U sys/i386/i386/genassym.c U sys/i386/i386/in_cksum.c U sys/i386/i386/locore.s C sys/i386/i386/machdep.c U sys/i386/i386/math_emu.h U sys/i386/i386/math_emulate.c U sys/i386/i386/mem.c U sys/i386/i386/microtime.s U sys/i386/i386/perfmon.c U sys/i386/i386/pmap.c U sys/i386/i386/procfs_machdep.c C sys/i386/i386/support.s U sys/i386/i386/swapgeneric.c U sys/i386/i386/swtch.s U sys/i386/i386/symbols.raw U sys/i386/i386/trap.c U sys/i386/i386/sys_machdep.c U sys/i386/i386/userconfig.c U sys/i386/i386/vm_machdep.c U sys/i386/i386/identcpu.c C sys/i386/i386/i386-gdbstub.c I sys/i386/ibcs2/CVS U sys/i386/ibcs2/coff.h U sys/i386/ibcs2/ibcs2_dirent.h U sys/i386/ibcs2/ibcs2_errno.c U sys/i386/ibcs2/ibcs2_errno.h U sys/i386/ibcs2/ibcs2_fcntl.c U sys/i386/ibcs2/ibcs2_fcntl.h U sys/i386/ibcs2/ibcs2_ioctl.c U sys/i386/ibcs2/ibcs2_ioctl.h U sys/i386/ibcs2/ibcs2_ipc.c U sys/i386/ibcs2/ibcs2_ipc.h U sys/i386/ibcs2/ibcs2_isc.c U sys/i386/ibcs2/ibcs2_isc_syscall.h U sys/i386/ibcs2/ibcs2_isc_sysent.c U sys/i386/ibcs2/ibcs2_misc.c U sys/i386/ibcs2/ibcs2_mount.h U sys/i386/ibcs2/ibcs2_msg.c U sys/i386/ibcs2/ibcs2_other.c U sys/i386/ibcs2/ibcs2_poll.h U sys/i386/ibcs2/ibcs2_proto.h U sys/i386/ibcs2/ibcs2_signal.c U sys/i386/ibcs2/ibcs2_signal.h U sys/i386/ibcs2/ibcs2_socksys.c U sys/i386/ibcs2/ibcs2_socksys.h U sys/i386/ibcs2/ibcs2_stat.c U sys/i386/ibcs2/ibcs2_stat.h U sys/i386/ibcs2/ibcs2_statfs.h U sys/i386/ibcs2/ibcs2_stropts.h U sys/i386/ibcs2/ibcs2_syscall.h U sys/i386/ibcs2/ibcs2_sysent.c U sys/i386/ibcs2/ibcs2_sysi86.c U sys/i386/ibcs2/ibcs2_sysvec.c U sys/i386/ibcs2/ibcs2_termios.h U sys/i386/ibcs2/ibcs2_time.h U sys/i386/ibcs2/ibcs2_types.h U sys/i386/ibcs2/ibcs2_unistd.h U sys/i386/ibcs2/ibcs2_ustat.h U sys/i386/ibcs2/ibcs2_util.c U sys/i386/ibcs2/ibcs2_util.h U sys/i386/ibcs2/ibcs2_utime.h U sys/i386/ibcs2/ibcs2_utsname.h U sys/i386/ibcs2/ibcs2_xenix.c U sys/i386/ibcs2/ibcs2_xenix.h U sys/i386/ibcs2/ibcs2_xenix_syscall.h U sys/i386/ibcs2/ibcs2_xenix_sysent.c U sys/i386/ibcs2/imgact_coff.c U sys/i386/ibcs2/syscalls.conf U sys/i386/ibcs2/syscalls.isc U sys/i386/ibcs2/syscalls.isc.conf U sys/i386/ibcs2/syscalls.master U sys/i386/ibcs2/syscalls.xenix U sys/i386/ibcs2/syscalls.xenix.conf I sys/i386/include/CVS U sys/i386/include/apm_bios.h U sys/i386/include/apm_segments.h U sys/i386/include/asc_ioctl.h U sys/i386/include/asmacros.h U sys/i386/include/bootinfo.h U sys/i386/include/clock.h U sys/i386/include/conf.h U sys/i386/include/cons.h U sys/i386/include/console.h U sys/i386/include/cpu.h U sys/i386/include/cpufunc.h U sys/i386/include/cputypes.h U sys/i386/include/cronyx.h U sys/i386/include/db_machdep.h U sys/i386/include/gsc.h U sys/i386/include/endian.h U sys/i386/include/exec.h U sys/i386/include/float.h U sys/i386/include/floatingpoint.h U sys/i386/include/frame.h U sys/i386/include/lpt.h U sys/i386/include/ieeefp.h U sys/i386/include/ioctl_ctx.h U sys/i386/include/ioctl_fd.h U sys/i386/include/ioctl_meteor.h U sys/i386/include/ipl.h U sys/i386/include/joystick.h U sys/i386/include/pcaudioio.h U sys/i386/include/limits.h U sys/i386/include/pmap.h U sys/i386/include/md_var.h U sys/i386/include/mouse.h U sys/i386/include/mtpr.h U sys/i386/include/npx.h U sys/i386/include/param.h U sys/i386/include/perfmon.h U sys/i386/include/pcb.h U sys/i386/include/pcvt_ioctl.h U sys/i386/include/random.h U sys/i386/include/types.h U sys/i386/include/proc.h U sys/i386/include/profile.h U sys/i386/include/psl.h U sys/i386/include/qcam.h U sys/i386/include/ptrace.h U sys/i386/include/spl.h U sys/i386/include/segments.h U sys/i386/include/reg.h U sys/i386/include/reloc.h U sys/i386/include/soundcard.h U sys/i386/include/si.h U sys/i386/include/signal.h U sys/i386/include/speaker.h U sys/i386/include/spigot.h U sys/i386/include/specialreg.h U sys/i386/include/trap.h U sys/i386/include/stdarg.h U sys/i386/include/sysarch.h U sys/i386/include/tss.h U sys/i386/include/ultrasound.h U sys/i386/include/varargs.h U sys/i386/include/vmparam.h U sys/i386/include/wtio.h U sys/i386/include/ansi.h U sys/i386/include/in_cksum.h U sys/i386/include/cdk.h U sys/i386/include/comstats.h I sys/i386/include/pc/CVS U sys/i386/include/pc/display.h U sys/i386/include/pc/msdos.h I sys/i386/isa/CVS U sys/i386/isa/README.le U sys/i386/isa/aic6360.c U sys/i386/isa/asc.c U sys/i386/isa/ascreg.h U sys/i386/isa/atapi.c U sys/i386/isa/atapi.h U sys/i386/isa/b004.c U sys/i386/isa/b004.h U sys/i386/isa/bt5xx-445.c U sys/i386/isa/clock.c U sys/i386/isa/cronyx.c U sys/i386/isa/ctx.c U sys/i386/isa/ctxreg.h U sys/i386/isa/cx.c U sys/i386/isa/cxreg.h U sys/i386/isa/cy.c U sys/i386/isa/cyreg.h U sys/i386/isa/diskslice_machdep.c U sys/i386/isa/elink.c U sys/i386/isa/elink.h U sys/i386/isa/fd.c U sys/i386/isa/fdc.h U sys/i386/isa/fdreg.h U sys/i386/isa/ft.c U sys/i386/isa/ftreg.h U sys/i386/isa/gpib.c U sys/i386/isa/gpib.h U sys/i386/isa/gpibreg.h U sys/i386/isa/gsc.c U sys/i386/isa/gscreg.h U sys/i386/isa/icu.h U sys/i386/isa/icu.s U sys/i386/isa/if_ar.c U sys/i386/isa/if_arregs.h U sys/i386/isa/if_cx.c U sys/i386/isa/if_ed.c U sys/i386/isa/if_edreg.h U sys/i386/isa/if_eg.c U sys/i386/isa/if_egreg.h U sys/i386/isa/if_el.c U sys/i386/isa/if_elreg.h U sys/i386/isa/if_ep.c U sys/i386/isa/if_epreg.h U sys/i386/isa/if_fe.c U sys/i386/isa/if_fereg.h U sys/i386/isa/if_ie.c U sys/i386/isa/if_ie507.h U sys/i386/isa/if_iereg.h U sys/i386/isa/if_ix.c U sys/i386/isa/if_ixreg.h U sys/i386/isa/if_le.c U sys/i386/isa/if_lnc.c U sys/i386/isa/if_lnc.h U sys/i386/isa/if_ze.c U sys/i386/isa/if_zp.c U sys/i386/isa/if_zpreg.h U sys/i386/isa/isa.c U sys/i386/isa/isa.h U sys/i386/isa/isa_device.h U sys/i386/isa/joy.c U sys/i386/isa/kbdio.c U sys/i386/isa/kbdtables.h U sys/i386/isa/labpc.c U sys/i386/isa/lpt.c U sys/i386/isa/lptreg.h U sys/i386/isa/mcd.c U sys/i386/isa/mcdreg.h U sys/i386/isa/mse.c U sys/i386/isa/ncr5380.c C sys/i386/isa/npx.c U sys/i386/isa/pcaudio.c U sys/i386/isa/pcibus.c U sys/i386/isa/pcic.h U sys/i386/isa/pcicx.c U sys/i386/isa/prof_machdep.c U sys/i386/isa/psm.c U sys/i386/isa/qcam.c U sys/i386/isa/qcamdefs.h U sys/i386/isa/qcamio.c U sys/i386/isa/qcamreg.h U sys/i386/isa/rc.c U sys/i386/isa/random_machdep.c U sys/i386/isa/rcreg.h U sys/i386/isa/rtc.h U sys/i386/isa/scd.c U sys/i386/isa/scdreg.h U sys/i386/isa/seagate.c U sys/i386/isa/si.c U sys/i386/isa/si_code.c C sys/i386/isa/sio.c U sys/i386/isa/sioreg.h U sys/i386/isa/sireg.h U sys/i386/isa/spigot.c U sys/i386/isa/spkr.c U sys/i386/isa/syscons.c U sys/i386/isa/syscons.h U sys/i386/isa/timerreg.h U sys/i386/isa/tw.c U sys/i386/isa/ultra14f.c U sys/i386/isa/vector.s U sys/i386/isa/wcd.c U sys/i386/isa/wd.c U sys/i386/isa/wd7000.c U sys/i386/isa/wdreg.h U sys/i386/isa/wt.c U sys/i386/isa/wtreg.h U sys/i386/isa/aha1542.c U sys/i386/isa/README.stl U sys/i386/isa/istallion.c U sys/i386/isa/stallion.c U sys/i386/isa/if_sr.c U sys/i386/isa/if_srregs.h U sys/i386/isa/adv_isa.c U sys/i386/isa/aic_98.h U sys/i386/isa/kbdio.h I sys/i386/isa/ic/CVS U sys/i386/isa/ic/Am7990.h U sys/i386/isa/ic/am7990.h U sys/i386/isa/ic/cd1400.h U sys/i386/isa/ic/cd180.h U sys/i386/isa/ic/esp.h U sys/i386/isa/ic/hd64570.h U sys/i386/isa/ic/i8042.h U sys/i386/isa/ic/i82365.h U sys/i386/isa/ic/i8237.h U sys/i386/isa/ic/i82586.h U sys/i386/isa/ic/lemac.h U sys/i386/isa/ic/mb86960.h U sys/i386/isa/ic/ncr53400.h U sys/i386/isa/ic/ncr5380.h U sys/i386/isa/ic/nec765.h U sys/i386/isa/ic/ns16450.h U sys/i386/isa/ic/ns16550.h U sys/i386/isa/ic/scd1400.h U sys/i386/isa/ic/i8251.h U sys/i386/isa/ic/wd33c93.h I sys/i386/isa/matcd/CVS U sys/i386/isa/matcd/TODO U sys/i386/isa/matcd/audio.c U sys/i386/isa/matcd/creative.h U sys/i386/isa/matcd/matcd.c U sys/i386/isa/matcd/matcddrv.h U sys/i386/isa/matcd/options.h I sys/i386/isa/pcvt/CVS U sys/i386/isa/pcvt/pcvt_conf.h U sys/i386/isa/pcvt/pcvt_drv.c U sys/i386/isa/pcvt/pcvt_ext.c U sys/i386/isa/pcvt/pcvt_hdr.h U sys/i386/isa/pcvt/pcvt_kbd.c U sys/i386/isa/pcvt/pcvt_kbd.h U sys/i386/isa/pcvt/pcvt_out.c U sys/i386/isa/pcvt/pcvt_sup.c U sys/i386/isa/pcvt/pcvt_tbl.h U sys/i386/isa/pcvt/pcvt_vtf.c I sys/i386/isa/sound/CVS U sys/i386/isa/sound/CHANGELOG U sys/i386/isa/sound/COPYING U sys/i386/isa/sound/README U sys/i386/isa/sound/Readme.aedsp16 U sys/i386/isa/sound/Readme.modules U sys/i386/isa/sound/Readme.v30 U sys/i386/isa/sound/ad1848.c U sys/i386/isa/sound/ad1848_mixer.h U sys/i386/isa/sound/adlib_card.c U sys/i386/isa/sound/aedsp16.c U sys/i386/isa/sound/audio.c U sys/i386/isa/sound/coproc.h U sys/i386/isa/sound/dev_table.c U sys/i386/isa/sound/dev_table.h U sys/i386/isa/sound/dmabuf.c U sys/i386/isa/sound/finetune.h U sys/i386/isa/sound/gus_card.c U sys/i386/isa/sound/gus_hw.h U sys/i386/isa/sound/gus_linearvol.h U sys/i386/isa/sound/gus_midi.c U sys/i386/isa/sound/gus_vol.c U sys/i386/isa/sound/gus_wave.c U sys/i386/isa/sound/hex2hex.h U sys/i386/isa/sound/ics2101.c U sys/i386/isa/sound/local.h U sys/i386/isa/sound/mad16.h U sys/i386/isa/sound/midi_ctrl.h U sys/i386/isa/sound/midi_synth.c U sys/i386/isa/sound/midi_synth.h U sys/i386/isa/sound/midibuf.c U sys/i386/isa/sound/mpu401.c U sys/i386/isa/sound/opl3.c U sys/i386/isa/sound/opl3.h U sys/i386/isa/sound/os.h U sys/i386/isa/sound/pas2_card.c U sys/i386/isa/sound/pas2_midi.c U sys/i386/isa/sound/pas2_mixer.c U sys/i386/isa/sound/pas2_pcm.c U sys/i386/isa/sound/patmgr.c U sys/i386/isa/sound/sb16_dsp.c U sys/i386/isa/sound/sb16_midi.c U sys/i386/isa/sound/sb_card.c U sys/i386/isa/sound/sb_dsp.c U sys/i386/isa/sound/sb_midi.c U sys/i386/isa/sound/sb_mixer.c U sys/i386/isa/sound/sb_mixer.h U sys/i386/isa/sound/sequencer.c U sys/i386/isa/sound/sound.doc U sys/i386/isa/sound/sound_calls.h U sys/i386/isa/sound/sound_config.h U sys/i386/isa/sound/sound_switch.c U sys/i386/isa/sound/sound_timer.c U sys/i386/isa/sound/soundcard.c U sys/i386/isa/sound/soundvers.h U sys/i386/isa/sound/sscape.c U sys/i386/isa/sound/sys_timer.c U sys/i386/isa/sound/trix.c U sys/i386/isa/sound/tuning.h U sys/i386/isa/sound/uart6850.c U sys/i386/isa/sound/ulaw.h U sys/i386/isa/sound/pcm86.c U sys/i386/isa/sound/awe_hw.h U sys/i386/isa/sound/awe_voice.h U sys/i386/isa/sound/awe_wave.c U sys/i386/isa/sound/pas_defs.h U sys/i386/isa/sound/sb_defs.h I sys/i386/linux/CVS U sys/i386/linux/imgact_linux.c U sys/i386/linux/linux.h U sys/i386/linux/linux_dummy.c U sys/i386/linux/linux_file.c U sys/i386/linux/linux_genassym.c U sys/i386/linux/linux_ioctl.c U sys/i386/linux/linux_ipc.c U sys/i386/linux/linux_locore.s U sys/i386/linux/linux_misc.c U sys/i386/linux/linux_proto.h U sys/i386/linux/linux_signal.c U sys/i386/linux/linux_socket.c U sys/i386/linux/linux_stats.c U sys/i386/linux/linux_syscall.h U sys/i386/linux/linux_sysent.c U sys/i386/linux/linux_sysvec.c U sys/i386/linux/linux_util.c U sys/i386/linux/linux_util.h U sys/i386/linux/syscalls.conf U sys/i386/linux/syscalls.master I sys/i386/scsi/CVS U sys/i386/scsi/93cx6.c U sys/i386/scsi/93cx6.h U sys/i386/scsi/aic7xxx.c U sys/i386/scsi/aic7xxx.h U sys/i386/scsi/bt.c U sys/i386/scsi/btreg.h U sys/i386/scsi/advansys.c U sys/i386/scsi/advansys.h I sys/isofs/CVS I sys/isofs/cd9660/CVS U sys/isofs/cd9660/TODO U sys/isofs/cd9660/TODO.hibler U sys/isofs/cd9660/cd9660_bmap.c U sys/isofs/cd9660/cd9660_lookup.c U sys/isofs/cd9660/cd9660_mount.h U sys/isofs/cd9660/cd9660_node.c U sys/isofs/cd9660/cd9660_node.h U sys/isofs/cd9660/cd9660_rrip.c U sys/isofs/cd9660/cd9660_rrip.h U sys/isofs/cd9660/cd9660_util.c U sys/isofs/cd9660/cd9660_vfsops.c U sys/isofs/cd9660/cd9660_vnops.c U sys/isofs/cd9660/iso.h U sys/isofs/cd9660/iso_rrip.h I sys/kern/CVS U sys/kern/Make.tags.inc U sys/kern/Makefile U sys/kern/imgact_aout.c U sys/kern/imgact_elf.c U sys/kern/imgact_gzip.c U sys/kern/imgact_shell.c U sys/kern/inflate.c U sys/kern/init_main.c U sys/kern/init_sysent.c U sys/kern/init_sysvec.c U sys/kern/kern_acct.c U sys/kern/kern_clock.c U sys/kern/kern_conf.c U sys/kern/kern_descrip.c U sys/kern/kern_exit.c U sys/kern/kern_exec.c U sys/kern/kern_mib.c U sys/kern/kern_fork.c U sys/kern/kern_ktrace.c U sys/kern/kern_lkm.c U sys/kern/kern_lockf.c U sys/kern/kern_malloc.c U sys/kern/kern_ntptime.c U sys/kern/kern_physio.c U sys/kern/kern_proc.c U sys/kern/kern_prot.c U sys/kern/kern_resource.c C sys/kern/kern_sig.c U sys/kern/kern_subr.c U sys/kern/kern_synch.c U sys/kern/kern_sysctl.c U sys/kern/kern_time.c U sys/kern/kern_xxx.c U sys/kern/makesyscalls.sh U sys/kern/subr_autoconf.c U sys/kern/subr_diskslice.c U sys/kern/subr_dkbad.c U sys/kern/subr_log.c U sys/kern/subr_prf.c U sys/kern/subr_prof.c U sys/kern/subr_rlist.c U sys/kern/subr_xxx.c U sys/kern/sys_pipe.c U sys/kern/sys_generic.c U sys/kern/sys_process.c U sys/kern/sys_socket.c U sys/kern/syscalls.c U sys/kern/syscalls.master U sys/kern/sysv_ipc.c U sys/kern/sysv_msg.c U sys/kern/sysv_sem.c U sys/kern/sysv_shm.c C sys/kern/tty.c U sys/kern/tty_compat.c U sys/kern/tty_conf.c U sys/kern/tty_pty.c U sys/kern/tty_snoop.c U sys/kern/tty_subr.c U sys/kern/tty_tb.c U sys/kern/tty_tty.c U sys/kern/uipc_domain.c U sys/kern/uipc_mbuf.c U sys/kern/uipc_proto.c U sys/kern/uipc_socket.c U sys/kern/uipc_socket2.c U sys/kern/uipc_syscalls.c U sys/kern/uipc_usrreq.c C sys/kern/vfs_bio.c U sys/kern/vfs_cache.c U sys/kern/vfs_cluster.c U sys/kern/vfs_conf.c U sys/kern/vfs_init.c U sys/kern/vfs_lookup.c U sys/kern/vfs_subr.c U sys/kern/vfs_syscalls.c U sys/kern/vfs_vnops.c U sys/kern/vnode_if.sh U sys/kern/vnode_if.src U sys/kern/kern_shutdown.c I sys/libkern/CVS U sys/libkern/adddi3.c U sys/libkern/anddi3.c U sys/libkern/ashldi3.c U sys/libkern/ashrdi3.c U sys/libkern/bcd.c U sys/libkern/bcmp.c U sys/libkern/cmpdi2.c U sys/libkern/divdi3.c U sys/libkern/ffs.c U sys/libkern/inet_ntoa.c U sys/libkern/iordi3.c U sys/libkern/locc.c U sys/libkern/lshldi3.c U sys/libkern/lshrdi3.c U sys/libkern/mcount.c U sys/libkern/moddi3.c U sys/libkern/muldi3.c U sys/libkern/negdi2.c U sys/libkern/notdi2.c U sys/libkern/qdivrem.c U sys/libkern/qsort.c U sys/libkern/quad.h U sys/libkern/random.c U sys/libkern/rindex.c U sys/libkern/scanc.c U sys/libkern/skpc.c U sys/libkern/strcat.c U sys/libkern/strcmp.c U sys/libkern/strcpy.c U sys/libkern/strlen.c U sys/libkern/strncmp.c U sys/libkern/strncpy.c U sys/libkern/subdi3.c U sys/libkern/ucmpdi2.c U sys/libkern/udivdi3.c U sys/libkern/umoddi3.c U sys/libkern/xordi3.c U sys/libkern/index.c I sys/miscfs/CVS I sys/miscfs/deadfs/CVS U sys/miscfs/deadfs/dead_vnops.c I sys/miscfs/devfs/CVS U sys/miscfs/devfs/README U sys/miscfs/devfs/devfs_proto.h U sys/miscfs/devfs/devfs_tree.c U sys/miscfs/devfs/devfs_vfsops.c U sys/miscfs/devfs/devfs_vnops.c U sys/miscfs/devfs/devfsdefs.h U sys/miscfs/devfs/reproto.sh I sys/miscfs/fdesc/CVS U sys/miscfs/fdesc/fdesc.h U sys/miscfs/fdesc/fdesc_vfsops.c U sys/miscfs/fdesc/fdesc_vnops.c I sys/miscfs/fifofs/CVS U sys/miscfs/fifofs/fifo.h U sys/miscfs/fifofs/fifo_vnops.c I sys/miscfs/kernfs/CVS U sys/miscfs/kernfs/kernfs.h U sys/miscfs/kernfs/kernfs_vfsops.c U sys/miscfs/kernfs/kernfs_vnops.c I sys/miscfs/nullfs/CVS U sys/miscfs/nullfs/null.h U sys/miscfs/nullfs/null_subr.c U sys/miscfs/nullfs/null_vfsops.c U sys/miscfs/nullfs/null_vnops.c I sys/miscfs/portal/CVS U sys/miscfs/portal/portal.h U sys/miscfs/portal/portal_vfsops.c U sys/miscfs/portal/portal_vnops.c I sys/miscfs/procfs/CVS U sys/miscfs/procfs/README U sys/miscfs/procfs/procfs.h U sys/miscfs/procfs/procfs_ctl.c U sys/miscfs/procfs/procfs_fpregs.c U sys/miscfs/procfs/procfs_mem.c U sys/miscfs/procfs/procfs_note.c U sys/miscfs/procfs/procfs_regs.c U sys/miscfs/procfs/procfs_status.c U sys/miscfs/procfs/procfs_subr.c U sys/miscfs/procfs/procfs_vfsops.c U sys/miscfs/procfs/procfs_vnops.c U sys/miscfs/procfs/procfs_map.c U sys/miscfs/procfs/procfs_type.c I sys/miscfs/specfs/CVS U sys/miscfs/specfs/spec_vnops.c U sys/miscfs/specfs/specdev.h I sys/miscfs/umapfs/CVS U sys/miscfs/umapfs/umap.h U sys/miscfs/umapfs/umap_subr.c U sys/miscfs/umapfs/umap_vfsops.c U sys/miscfs/umapfs/umap_vnops.c I sys/miscfs/union/CVS U sys/miscfs/union/README U sys/miscfs/union/libc.fts.c U sys/miscfs/union/libc.opendir.c U sys/miscfs/union/libc.readdir.c U sys/miscfs/union/union.h U sys/miscfs/union/union_subr.c U sys/miscfs/union/union_vfsops.c U sys/miscfs/union/union_vnops.c I sys/msdosfs/CVS U sys/msdosfs/bootsect.h U sys/msdosfs/bpb.h U sys/msdosfs/denode.h U sys/msdosfs/direntry.h U sys/msdosfs/fat.h U sys/msdosfs/msdosfs_conv.c U sys/msdosfs/msdosfs_denode.c U sys/msdosfs/msdosfs_fat.c U sys/msdosfs/msdosfs_lookup.c U sys/msdosfs/msdosfs_vfsops.c U sys/msdosfs/msdosfs_vnops.c U sys/msdosfs/msdosfsmount.h I sys/net/CVS U sys/net/bpf.c U sys/net/bpf.h U sys/net/bpf_compat.h U sys/net/bpf_filter.c U sys/net/bpfdesc.h U sys/net/bsd_comp.c U sys/net/if.c U sys/net/if.h U sys/net/if_arp.h U sys/net/if_disc.c U sys/net/if_dl.h U sys/net/if_ethersubr.c U sys/net/if_fddisubr.c U sys/net/if_llc.h U sys/net/if_loop.c U sys/net/if_ppp.c U sys/net/if_ppp.h U sys/net/if_pppvar.h U sys/net/if_sl.c U sys/net/if_slvar.h U sys/net/if_sppp.h U sys/net/if_spppsubr.c U sys/net/if_tun.c U sys/net/if_tun.h U sys/net/if_types.h U sys/net/netisr.h U sys/net/ppp_comp.h U sys/net/ppp_defs.h U sys/net/ppp_tty.c U sys/net/radix.c U sys/net/radix.h U sys/net/raw_cb.c U sys/net/raw_cb.h U sys/net/raw_usrreq.c U sys/net/route.c U sys/net/route.h U sys/net/rtsock.c U sys/net/slcompress.c U sys/net/slcompress.h U sys/net/slip.h U sys/net/ethernet.h U sys/net/if_mib.c U sys/net/if_mib.h I sys/netinet/CVS U sys/netinet/icmp_var.h U sys/netinet/if_ether.c U sys/netinet/if_ether.h U sys/netinet/if_fddi.h U sys/netinet/igmp.c U sys/netinet/igmp.h U sys/netinet/igmp_var.h U sys/netinet/in.c U sys/netinet/in.h U sys/netinet/in_cksum.c U sys/netinet/in_pcb.c U sys/netinet/in_pcb.h U sys/netinet/in_proto.c U sys/netinet/in_rmx.c U sys/netinet/in_systm.h U sys/netinet/in_var.h U sys/netinet/ip.h U sys/netinet/ip_fw.c U sys/netinet/ip_fw.h U sys/netinet/ip_icmp.c U sys/netinet/ip_icmp.h U sys/netinet/ip_input.c U sys/netinet/ip_mroute.c U sys/netinet/ip_mroute.h U sys/netinet/ip_output.c U sys/netinet/ip_var.h U sys/netinet/raw_ip.c U sys/netinet/tcp.h U sys/netinet/tcp_debug.c U sys/netinet/tcp_debug.h U sys/netinet/tcp_fsm.h U sys/netinet/tcp_input.c U sys/netinet/tcp_output.c U sys/netinet/tcp_seq.h U sys/netinet/tcp_subr.c U sys/netinet/tcp_timer.c U sys/netinet/tcp_timer.h U sys/netinet/tcp_usrreq.c U sys/netinet/tcp_var.h U sys/netinet/tcpip.h U sys/netinet/udp.h U sys/netinet/udp_usrreq.c U sys/netinet/udp_var.h U sys/netinet/ip_divert.c I sys/netipx/CVS U sys/netipx/README U sys/netipx/ipx.c U sys/netipx/ipx.h U sys/netipx/ipx_cksum.c U sys/netipx/ipx_error.c U sys/netipx/ipx_error.h U sys/netipx/ipx_if.h U sys/netipx/ipx_input.c U sys/netipx/ipx_ip.c U sys/netipx/ipx_ip.h U sys/netipx/ipx_outputfl.c U sys/netipx/ipx_pcb.c U sys/netipx/ipx_pcb.h U sys/netipx/ipx_proto.c U sys/netipx/ipx_tun.c U sys/netipx/ipx_usrreq.c U sys/netipx/ipx_var.h U sys/netipx/spx.h U sys/netipx/spx_debug.c U sys/netipx/spx_debug.h U sys/netipx/spx_timer.h U sys/netipx/spx_usrreq.c U sys/netipx/spx_var.h I sys/nfs/CVS U sys/nfs/nfs.h U sys/nfs/nfs_bio.c U sys/nfs/nfs_node.c U sys/nfs/nfs_nqlease.c U sys/nfs/nfs_serv.c U sys/nfs/nfs_socket.c U sys/nfs/nfs_srvcache.c U sys/nfs/nfs_subs.c U sys/nfs/nfs_syscalls.c U sys/nfs/nfs_vfsops.c U sys/nfs/nfs_vnops.c U sys/nfs/nfsdiskless.h U sys/nfs/nfsm_subs.h U sys/nfs/nfsmount.h U sys/nfs/nfsnode.h U sys/nfs/nfsproto.h U sys/nfs/nfsrtt.h U sys/nfs/nfsrvcache.h U sys/nfs/nfsv2.h U sys/nfs/nqnfs.h U sys/nfs/rpcv2.h U sys/nfs/xdr_subs.h I sys/pccard/CVS U sys/pccard/card.h U sys/pccard/cis.h U sys/pccard/driver.h U sys/pccard/i82365.h U sys/pccard/pccard.c U sys/pccard/pcic.c U sys/pccard/skel.c U sys/pccard/slot.h U sys/pccard/pcic98reg.h I sys/pci/CVS U sys/pci/README.de U sys/pci/README.de-le U sys/pci/aic7870.c U sys/pci/bt9xx.c U sys/pci/dc21040.h U sys/pci/if_de.c U sys/pci/if_fxp.c U sys/pci/if_fxpreg.h U sys/pci/if_pdq.c U sys/pci/meteor.c U sys/pci/ncr.c U sys/pci/locate.pl U sys/pci/ncrreg.h U sys/pci/meteor_reg.h U sys/pci/pci.c U sys/pci/pci_ioctl.h U sys/pci/pcibus.h U sys/pci/pcireg.h U sys/pci/pcisupport.c U sys/pci/pcivar.h U sys/pci/pdq.c U sys/pci/pdq_os.h U sys/pci/pdqreg.h U sys/pci/wd82371.c U sys/pci/wd82371reg.h U sys/pci/if_ed_p.c U sys/pci/if_lnc_p.c U sys/pci/cy_pci.c U sys/pci/cy_pcireg.h U sys/pci/if_sr_p.c U sys/pci/if_vx_pci.c I sys/scsi/CVS U sys/scsi/README U sys/scsi/cd.c U sys/scsi/ch.c U sys/scsi/od.c U sys/scsi/pt.c U sys/scsi/scsi_all.h U sys/scsi/scsi_base.c U sys/scsi/scsi_cd.h U sys/scsi/scsi_changer.h U sys/scsi/scsi_debug.h U sys/scsi/scsi_disk.h U sys/scsi/scsi_driver.c U sys/scsi/scsi_driver.h U sys/scsi/scsi_generic.h U sys/scsi/scsi_ioctl.c U sys/scsi/scsi_sense.c U sys/scsi/scsi_tape.h U sys/scsi/scsi_worm.h U sys/scsi/scsiconf.c U sys/scsi/scsiconf.h U sys/scsi/sctarg.c C sys/scsi/sd.c U sys/scsi/ssc.c U sys/scsi/st.c U sys/scsi/su.c U sys/scsi/uk.c U sys/scsi/worm.c U sys/scsi/scsi_message.h I sys/sys/CVS U sys/sys/acct.h U sys/sys/buf.h U sys/sys/callout.h U sys/sys/cdefs.h U sys/sys/cdio.h U sys/sys/chio.h U sys/sys/clist.h U sys/sys/conf.h U sys/sys/dataacq.h U sys/sys/disklabel.h U sys/sys/devfsext.h U sys/sys/device.h U sys/sys/dir.h U sys/sys/dirent.h U sys/sys/disk.h U sys/sys/diskslice.h U sys/sys/dkbad.h U sys/sys/ftape.h U sys/sys/dkstat.h U sys/sys/dmap.h U sys/sys/domain.h U sys/sys/errno.h U sys/sys/exec.h U sys/sys/fbio.h U sys/sys/fcntl.h U sys/sys/file.h U sys/sys/filedesc.h U sys/sys/filio.h U sys/sys/ioctl.h U sys/sys/gmon.h U sys/sys/imgact.h U sys/sys/imgact_aout.h U sys/sys/imgact_elf.h U sys/sys/inflate.h U sys/sys/ioccom.h U sys/sys/mount.h U sys/sys/ioctl_compat.h U sys/sys/ipc.h U sys/sys/kernel.h U sys/sys/ktrace.h U sys/sys/libkern.h U sys/sys/lkm.h U sys/sys/lockf.h U sys/sys/malloc.h U sys/sys/mbuf.h U sys/sys/mman.h U sys/sys/mtio.h U sys/sys/msg.h U sys/sys/msgbuf.h U sys/sys/proc.h U sys/sys/namei.h U sys/sys/param.h U sys/sys/pipe.h U sys/sys/scsiio.h U sys/sys/protosw.h U sys/sys/ptrace.h U sys/sys/queue.h U sys/sys/reboot.h U sys/sys/resource.h U sys/sys/resourcevar.h U sys/sys/rlist.h U sys/sys/rtprio.h U sys/sys/sockio.h U sys/sys/select.h U sys/sys/sem.h U sys/sys/shm.h U sys/sys/signal.h U sys/sys/signalvar.h U sys/sys/snoop.h U sys/sys/socket.h U sys/sys/socketvar.h U sys/sys/stat.h U sys/sys/syscall-hide.h U sys/sys/syscall.h U sys/sys/sysctl.h U sys/sys/sysproto.h U sys/sys/sysent.h U sys/sys/syslimits.h U sys/sys/syslog.h U sys/sys/systm.h U sys/sys/tablet.h U sys/sys/time.h U sys/sys/termios.h U sys/sys/user.h U sys/sys/timeb.h U sys/sys/timers.h U sys/sys/times.h U sys/sys/timex.h U sys/sys/tprintf.h U sys/sys/tty.h U sys/sys/ttychars.h U sys/sys/ttycom.h U sys/sys/ttydefaults.h U sys/sys/ttydev.h U sys/sys/types.h U sys/sys/ucred.h U sys/sys/uio.h U sys/sys/un.h U sys/sys/unistd.h U sys/sys/unpcb.h U sys/sys/vcmd.h U sys/sys/utsname.h U sys/sys/vadvise.h U sys/sys/vnode.h U sys/sys/vlimit.h U sys/sys/vmmeter.h U sys/sys/vnioctl.h U sys/sys/vsio.h U sys/sys/wait.h U sys/sys/wormio.h U sys/sys/ccdvar.h I sys/ufs/CVS I sys/ufs/ffs/CVS U sys/ufs/ffs/ffs_alloc.c U sys/ufs/ffs/ffs_balloc.c U sys/ufs/ffs/ffs_extern.h U sys/ufs/ffs/ffs_inode.c U sys/ufs/ffs/ffs_subr.c U sys/ufs/ffs/ffs_tables.c U sys/ufs/ffs/ffs_vfsops.c U sys/ufs/ffs/ffs_vnops.c U sys/ufs/ffs/fs.h I sys/ufs/lfs/CVS U sys/ufs/lfs/README U sys/ufs/lfs/TODO U sys/ufs/lfs/lfs.h U sys/ufs/lfs/lfs_alloc.c U sys/ufs/lfs/lfs_balloc.c U sys/ufs/lfs/lfs_bio.c U sys/ufs/lfs/lfs_cksum.c U sys/ufs/lfs/lfs_debug.c U sys/ufs/lfs/lfs_extern.h U sys/ufs/lfs/lfs_inode.c U sys/ufs/lfs/lfs_segment.c U sys/ufs/lfs/lfs_subr.c U sys/ufs/lfs/lfs_syscalls.c U sys/ufs/lfs/lfs_vfsops.c U sys/ufs/lfs/lfs_vnops.c I sys/ufs/mfs/CVS U sys/ufs/mfs/mfs_extern.h U sys/ufs/mfs/mfs_vfsops.c U sys/ufs/mfs/mfs_vnops.c U sys/ufs/mfs/mfsiom.h U sys/ufs/mfs/mfsnode.h I sys/ufs/ufs/CVS U sys/ufs/ufs/dinode.h U sys/ufs/ufs/dir.h U sys/ufs/ufs/inode.h U sys/ufs/ufs/quota.h U sys/ufs/ufs/ufs_bmap.c U sys/ufs/ufs/ufs_disksubr.c U sys/ufs/ufs/ufs_extern.h U sys/ufs/ufs/ufs_ihash.c U sys/ufs/ufs/ufs_inode.c U sys/ufs/ufs/ufs_lookup.c U sys/ufs/ufs/ufs_quota.c U sys/ufs/ufs/ufs_readwrite.c U sys/ufs/ufs/ufs_vfsops.c U sys/ufs/ufs/ufs_vnops.c U sys/ufs/ufs/ufsmount.h I sys/vm/CVS U sys/vm/default_pager.c U sys/vm/default_pager.h U sys/vm/device_pager.c U sys/vm/device_pager.h U sys/vm/kern_lock.c U sys/vm/lock.h U sys/vm/pmap.h U sys/vm/swap_pager.c U sys/vm/swap_pager.h U sys/vm/vm.h U sys/vm/vm_extern.h U sys/vm/vm_fault.c U sys/vm/vm_glue.c U sys/vm/vm_inherit.h U sys/vm/vm_init.c U sys/vm/vm_kern.c U sys/vm/vm_kern.h U sys/vm/vm_map.c U sys/vm/vm_map.h U sys/vm/vm_meter.c U sys/vm/vm_mmap.c U sys/vm/vm_object.c U sys/vm/vm_object.h U sys/vm/vm_page.c U sys/vm/vm_page.h U sys/vm/vm_pageout.c U sys/vm/vm_pageout.h U sys/vm/vm_pager.c U sys/vm/vm_pager.h U sys/vm/vm_param.h U sys/vm/vm_prot.h U sys/vm/vm_swap.c U sys/vm/vm_unix.c U sys/vm/vnode_pager.c U sys/vm/vnode_pager.h I sys/netatalk/CVS U sys/netatalk/aarp.c U sys/netatalk/aarp.h U sys/netatalk/at.h U sys/netatalk/at_control.c U sys/netatalk/at_extern.h U sys/netatalk/at_proto.c U sys/netatalk/at_rmx.c U sys/netatalk/at_var.h U sys/netatalk/ddp.h U sys/netatalk/ddp_input.c U sys/netatalk/ddp_output.c U sys/netatalk/ddp_usrreq.c U sys/netatalk/ddp_var.h U sys/netatalk/endian.h U sys/netatalk/phase2.h U sys/netatalk/COPYRIGHT I sys/netns/CVS U sys/netns/idp.h U sys/netns/idp_usrreq.c U sys/netns/idp_var.h U sys/netns/ns.c U sys/netns/ns.h U sys/netns/ns_cksum.c U sys/netns/ns_error.c U sys/netns/ns_error.h U sys/netns/ns_if.h U sys/netns/ns_input.c U sys/netns/ns_ip.c U sys/netns/ns_output.c U sys/netns/ns_pcb.c U sys/netns/ns_pcb.h U sys/netns/ns_proto.c U sys/netns/sp.h U sys/netns/spidp.h U sys/netns/spp_debug.c U sys/netns/spp_debug.h U sys/netns/spp_timer.h U sys/netns/spp_usrreq.c U sys/netns/spp_var.h I sys/pc98/CVS I sys/pc98/boot/CVS U sys/pc98/boot/Makefile I sys/pc98/boot/biosboot/CVS U sys/pc98/boot/biosboot/Makefile U sys/pc98/boot/biosboot/README.386BSD U sys/pc98/boot/biosboot/README.MACH U sys/pc98/boot/biosboot/README.serial U sys/pc98/boot/biosboot/README.serial.98 U sys/pc98/boot/biosboot/asm.S U sys/pc98/boot/biosboot/asm.h U sys/pc98/boot/biosboot/bios.S U sys/pc98/boot/biosboot/boot.c U sys/pc98/boot/biosboot/boot.h U sys/pc98/boot/biosboot/boot2.S U sys/pc98/boot/biosboot/disk.c U sys/pc98/boot/biosboot/io.c U sys/pc98/boot/biosboot/probe_keyboard.c U sys/pc98/boot/biosboot/serial.S U sys/pc98/boot/biosboot/start.S U sys/pc98/boot/biosboot/sys.c U sys/pc98/boot/biosboot/table.c I sys/pc98/boot/kzipboot/CVS U sys/pc98/boot/kzipboot/Makefile U sys/pc98/boot/kzipboot/README U sys/pc98/boot/kzipboot/boot.c U sys/pc98/boot/kzipboot/gzip.h U sys/pc98/boot/kzipboot/head.S U sys/pc98/boot/kzipboot/malloc.c U sys/pc98/boot/kzipboot/misc.c U sys/pc98/boot/kzipboot/tail.S U sys/pc98/boot/kzipboot/unzip.c I sys/pc98/boot/netboot/CVS U sys/pc98/boot/netboot/3c509.c U sys/pc98/boot/netboot/3c509.h U sys/pc98/boot/netboot/Makefile U sys/pc98/boot/netboot/bootmenu.c U sys/pc98/boot/netboot/ether.c U sys/pc98/boot/netboot/ether.h U sys/pc98/boot/netboot/if_epreg.h U sys/pc98/boot/netboot/main.c U sys/pc98/boot/netboot/makerom.c U sys/pc98/boot/netboot/misc.c U sys/pc98/boot/netboot/netboot.doc U sys/pc98/boot/netboot/netboot.h U sys/pc98/boot/netboot/ns8390.c U sys/pc98/boot/netboot/ns8390.h U sys/pc98/boot/netboot/rpc.c U sys/pc98/boot/netboot/start2.S I sys/pc98/boot/rawboot/CVS U sys/pc98/boot/rawboot/Makefile U sys/pc98/boot/rawboot/README I sys/pc98/conf/CVS U sys/pc98/conf/GENERIC98 U sys/pc98/conf/Makefile.pc98 U sys/pc98/conf/devices.pc98 U sys/pc98/conf/files.pc98 U sys/pc98/conf/majors.pc98 U sys/pc98/conf/options.pc98 I sys/pc98/i386/CVS U sys/pc98/i386/locore.s U sys/pc98/i386/machdep.c U sys/pc98/i386/microtime.s U sys/pc98/i386/trap.c U sys/pc98/i386/userconfig.c I sys/pc98/pc98/CVS U sys/pc98/pc98/30line.h U sys/pc98/pc98/atcompat_diskslice.c U sys/pc98/pc98/clock.c U sys/pc98/pc98/diskslice_machdep.c U sys/pc98/pc98/epsonio.h U sys/pc98/pc98/fd.c U sys/pc98/pc98/fdreg.h U sys/pc98/pc98/ft.c U sys/pc98/pc98/if_ed.c U sys/pc98/pc98/if_ed98.h U sys/pc98/pc98/if_fe.c U sys/pc98/pc98/kbd.h U sys/pc98/pc98/lpt.c U sys/pc98/pc98/module.h U sys/pc98/pc98/mse.c U sys/pc98/pc98/npx.c U sys/pc98/pc98/pc98.c U sys/pc98/pc98/pc98.h U sys/pc98/pc98/pc98_machdep.c U sys/pc98/pc98/pc98_machdep.h U sys/pc98/pc98/pcaudio.c U sys/pc98/pc98/sbic55.c U sys/pc98/pc98/sbic55.c.new U sys/pc98/pc98/sbicreg.h U sys/pc98/pc98/sbicvar.h U sys/pc98/pc98/scsireg.h U sys/pc98/pc98/sio.c U sys/pc98/pc98/sioreg.h U sys/pc98/pc98/spkr.c U sys/pc98/pc98/syscons.c U sys/pc98/pc98/syscons.h U sys/pc98/pc98/wd.c 11 conflicts created by this import. Use the following command to help the merge: cvs checkout -jCURRENT:yesterday -jCURRENT sys From owner-freebsd-smp Tue Dec 3 03:50:35 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA04775 for smp-outgoing; Tue, 3 Dec 1996 03:50:35 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id DAA04762 for ; Tue, 3 Dec 1996 03:50:26 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id HAA12878; Tue, 3 Dec 1996 07:46:51 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612031146.HAA12878@bluenose.na.tuns.ca> Subject: Re: cvs commit: sys/i386/include spl.h To: smp@csn.net (Steve Passe) Date: Tue, 3 Dec 1996 07:46:51 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612030527.WAA25776@clem.systemsix.com> from Steve Passe at "Dec 2, 96 10:27:57 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > screwy in the "autostart" code. This is now off by default and the SMP-current > > > kernel is stable at this point. I am very curious to here from one of the EIDE > > > testers to see if we fixed that problem also... > > > > > I tested. The system can boot without any problem but > > if the second cpu is launched, the coredump occurrs to almost > > every binary. If I boot the system with the SCSI and fsck IDE, there > > are lots of DUP's.... The filesystem is screwed up (I believe). > > does the SCSI system work properly with the 2nd CPU? > Yes. Jim From owner-freebsd-smp Tue Dec 3 03:59:42 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA05250 for smp-outgoing; Tue, 3 Dec 1996 03:59:42 -0800 (PST) Received: from minnow.render.com (render.demon.co.uk [158.152.30.118]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id DAA05243 for ; Tue, 3 Dec 1996 03:59:37 -0800 (PST) Received: from minnow.render.com (minnow.render.com [193.195.178.1]) by minnow.render.com (8.6.12/8.6.9) with SMTP id LAA27483; Tue, 3 Dec 1996 11:58:31 GMT Date: Tue, 3 Dec 1996 11:58:31 +0000 (GMT) From: Doug Rabson To: "John S. Dyson" cc: Poul-Henning Kamp , smp@csn.net, freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/include pmap.h In-Reply-To: <199612022247.RAA00795@dyson.iquest.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk On Mon, 2 Dec 1996, John S. Dyson wrote: > > > > smp -------------- > > \ > > lite2 --------- \ > > \ \ > > current-----------------> > > > > But I'm pretty sure that lite2 will get merged before smp as things > > stand now. > > > I think that this will have to be the way that it is done. Additionally, > as the pmap/vm changes for smp are merged in, I hope to be able to > minimize the differences -- so that we minimize the complexity of > support. Certainly, we don't want to clobber our single processor > performance -- but yet we don't want to sacrifice any advantages of > the SMP configurations. Darn'it I have yet to review the SMP code, > but will probably do so tonight. At least it'll give me an idea of > what is going on. I hope that most of the differences are in the > interrupt/exception and initialization areas. > > Regarding Lite/2, I believe that DG, PHK or I will take an initial cut > on it (the work is from Jeff Hsu.) The stuff has aged a bit (perhaps > alot.) We'll all review/work on it, using each other as resources. > I don't know, maybe, if I continue to feel well from my vacation, I > can work on it a bit. I'll try to commit (wishy-washy, I know) to > doing something by the end of the week (fat chance) -- those kinds of > promises from me usually mean next week sometime... > > So much to do, and only 24 hours to do it in... The lite2 tree is in pretty good shape. The only problem with it at the moment is that it badly needs a merge with -current (and someone with the time to fix it after the merge) and it *really* badly needs someone other than me and Jeffrey to run it. I don't know about Jeffrey but I have virtually no time to spend on FreeBSD at the moment due to other commitments and lite2 is suffering as a result. -- Doug Rabson, Microsoft RenderMorphics Ltd. Mail: dfr@render.com Phone: +44 171 734 3761 FAX: +44 171 734 6426 From owner-freebsd-smp Tue Dec 3 04:22:42 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id EAA08089 for smp-outgoing; Tue, 3 Dec 1996 04:22:42 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id EAA08081 for freebsd-smp; Tue, 3 Dec 1996 04:22:37 -0800 (PST) Date: Tue, 3 Dec 1996 04:22:37 -0800 (PST) From: Peter Wemm Message-Id: <199612031222.EAA08081@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/conf files.i386 sys/i386/i386 machdep.c support.s sys/kern kern_sig.c tty.c vfs_bio.c sys/scsi sd.c sys/conf newvers.sh sys/i386/isa npx.c sio.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 04:22:35 Modified: conf newvers.sh i386/conf files.i386 i386/i386 machdep.c support.s i386/isa npx.c sio.c kern kern_sig.c tty.c vfs_bio.c scsi sd.c Log: Merge -current changes onto touched files on the smp mainline Revision Changes Path 1.3 +2 -2 sys/conf/newvers.sh 1.13 +1 -1 sys/i386/conf/files.i386 1.32 +5 -9 sys/i386/i386/machdep.c 1.15 +48 -32 sys/i386/i386/support.s 1.14 +1 -3 sys/i386/isa/npx.c 1.12 +79 -39 sys/i386/isa/sio.c 1.7 +9 -5 sys/kern/kern_sig.c 1.3 +30 -20 sys/kern/tty.c 1.12 +93 -20 sys/kern/vfs_bio.c 1.9 +81 -8 sys/scsi/sd.c From owner-freebsd-smp Tue Dec 3 06:05:12 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA16059 for smp-outgoing; Tue, 3 Dec 1996 06:05:12 -0800 (PST) Received: from gargoyle.bazzle.com (gargoyle.bazzle.com [206.103.246.190]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA16048 for ; Tue, 3 Dec 1996 06:05:04 -0800 (PST) Received: from gargoyle.bazzle.com (localhost [127.0.0.1]) by gargoyle.bazzle.com (8.8.3/8.6.12) with SMTP id JAA04237 for ; Tue, 3 Dec 1996 09:05:10 -0500 (EST) Date: Tue, 3 Dec 1996 09:05:10 -0500 (EST) From: "Eric J. Chet" To: smp@freebsd.org Subject: performance In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Mon, 2 Dec 1996, Eric J. Chet wrote: > Kernel compile times: > > smp-kernel compiled under -current (as of 12/01) > >time make > > 314.39s real 282.00s user 18.72s system > > smp-kernel compiled under -smp-current > >time make -j8 > > 213.57s real 338.08s user 77.12s system > Hello Just looking at the numbers again, food for thought. (u+s)/r == utilization. -current (282.00+18.72)/314.39 = .96 -smp-current (338.08+77.12)/213.57 = 1.94 !!! very good! I know there is a lot of development work to be done, but -smp is definitly on the right path. Eric J. Chet - ejc@bazzle.com From owner-freebsd-smp Tue Dec 3 06:38:35 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA17659 for smp-outgoing; Tue, 3 Dec 1996 06:38:35 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA17654 for ; Tue, 3 Dec 1996 06:38:26 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id WAA04281; Tue, 3 Dec 1996 22:37:33 +0800 (WST) Message-Id: <199612031437.WAA04281@spinner.DIALix.COM> To: "Eric J. Chet" cc: smp@freebsd.org Subject: Re: performance In-reply-to: Your message of "Tue, 03 Dec 1996 09:05:10 EST." Date: Tue, 03 Dec 1996 22:37:33 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk "Eric J. Chet" wrote: > > On Mon, 2 Dec 1996, Eric J. Chet wrote: > > > Kernel compile times: > > > > smp-kernel compiled under -current (as of 12/01) > > >time make > > > 314.39s real 282.00s user 18.72s system > > > > smp-kernel compiled under -smp-current > > >time make -j8 > > > 213.57s real 338.08s user 77.12s system > > > > Hello > Just looking at the numbers again, food for thought. > > (u+s)/r == utilization. > > -current > (282.00+18.72)/314.39 = .96 > > -smp-current > (338.08+77.12)/213.57 = 1.94 !!! very good! > > I know there is a lot of development work to be done, but -smp is > definitly on the right path. Actually, those stats are biased and can't be trusted. The only thing you can really count on is the "elapsed" time improvement, especially since our u+s accounting is somewhat bogus at present... (ie: what one cpu is doing is accredited to all currently running processes, from memory). What's a much fairer indication is: 314.39/213.57 = 1.47 Not quite as good, but still a way to go. > Eric J. Chet > - ejc@bazzle.com Cheers, -Peter From owner-freebsd-smp Tue Dec 3 10:58:37 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA05441 for smp-outgoing; Tue, 3 Dec 1996 10:58:37 -0800 (PST) Received: from ceylon.informatik.uni-rostock.de (ceylon.informatik.uni-rostock.de [139.30.5.237]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id KAA05427 for ; Tue, 3 Dec 1996 10:58:32 -0800 (PST) Received: by ceylon.informatik.uni-rostock.de id TAA07736; Tue, 3 Dec 1996 19:56:55 +0100 Received: by donau.informatik.uni-rostock.de id TAA11295; Tue, 3 Dec 1996 19:56:51 +0100 (MET) Date: Tue, 3 Dec 1996 19:56:51 +0100 (MET) From: Gunther Hipper Message-Id: <199612031856.TAA11295@donau.informatik.uni-rostock.de> To: freebsd-smp@freebsd.org Subject: Great ! Cc: smp@csn.net, gunther@ceylon.informatik.uni-rostock.de X-Sun-Charset: US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi all ! This is great news for me - since 19:50 MET the SMP kernel is __UP__ and __RUNNING__ ! No more IDE-problems ! Thank you all and especially Steve ! What was the reason ? I'll now start doing my benchmarking ... tonight i'll (hopefully) get my network-driver and the FORTRAN compiler up and running. If you're interested, i'll post a picture of the PC-cluster running FreeBSD and the benchmark results ! Thanks Gunther P.S.: Steve - the serial terminal is on the way, I promise ! Gunther Hipper | University of Rostock | Department of Computer Science | Tel +49 381 498 3391 Albert-Einstein-Str. 21 | Fax +49 381 498 3440 18051 Rostock - Germany | Email Gunther.Hipper@informatik.uni-rostock.de From owner-freebsd-smp Tue Dec 3 11:09:27 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA06058 for smp-outgoing; Tue, 3 Dec 1996 11:09:27 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA06053 for ; Tue, 3 Dec 1996 11:09:24 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id MAA29775; Tue, 3 Dec 1996 12:06:22 -0700 Message-Id: <199612031906.MAA29775@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Gunther Hipper cc: freebsd-smp@freebsd.org Subject: Re: Great ! In-reply-to: Your message of "Tue, 03 Dec 1996 19:56:51 +0100." <199612031856.TAA11295@donau.informatik.uni-rostock.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Dec 1996 12:06:22 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Gunther, > This is great news for me - since 19:50 MET the SMP kernel is > __UP__ and __RUNNING__ ! No more IDE-problems ! its even greater news to me, I was about to go out and buy an IDE drive to try and figure out what the f*^& was happening! Important: for the record, do you have APIC_IO and IDE working? --- > What was the reason ? it was *caused* by our attempts to autostart the 2nd CPU, as to the *reason*, I'd consider giving up my right arm to know why... -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Tue Dec 3 11:39:56 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA07218 for smp-outgoing; Tue, 3 Dec 1996 11:39:56 -0800 (PST) Received: from uruk.org (ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA07178 for ; Tue, 3 Dec 1996 11:38:39 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vV0hg-0008Ap-00; Tue, 3 Dec 1996 11:40:28 -0800 To: smp@csn.net cc: smp@freebsd.org Subject: APIC_IO stuff Date: Tue, 03 Dec 1996 11:40:28 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I haven't had much time to try to solve the syscons problem, so I've been running SMP by remote login over ethernet. Last night I tried an interesting experiment, just run the APIC_IO version but don't start the second CPU. It has run all night and through til noon today with no problems (I did a few moderate compiles). P.S.: When I mentioned all the CPUs being started up at some earlier date, I may have been smoking something. It only does anything with one of the other CPUs in my test box since I've looked carefully at it again. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Tue Dec 3 11:44:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA07496 for smp-outgoing; Tue, 3 Dec 1996 11:44:46 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id LAA07491 for ; Tue, 3 Dec 1996 11:44:43 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id MAA00118; Tue, 3 Dec 1996 12:44:20 -0700 Message-Id: <199612031944.MAA00118@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Erich Boleyn cc: smp@freebsd.org Subject: Re: APIC_IO stuff In-reply-to: Your message of "Tue, 03 Dec 1996 11:40:28 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Dec 1996 12:44:20 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Erich, > I haven't had much time to try to solve the syscons problem, so I've > been running SMP by remote login over ethernet. > > Last night I tried an interesting experiment, just run the APIC_IO > version but don't start the second CPU. It has run all night and > through til noon today with no problems (I did a few moderate compiles). resup and retry, the version you have was broken big-time. you should be able to run 2 CPUS & APIC_IO without problem. If syscons is still a problem you could try using pcvt instead. --- > P.S.: When I mentioned all the CPUs being started up at some earlier > date, I may have been smoking something. It only does anything > with one of the other CPUs in my test box since I've looked > carefully at it again. > this is kinda good, it means your board is acting consistantly with the other known 4 CPU board being tested. 4 CPU support is off again, but we are probably pretty close to attempting it again, just will require a manual start of the APs. Peter could better decsribe where we stand on this issue. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Tue Dec 3 12:15:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA08796 for smp-outgoing; Tue, 3 Dec 1996 12:15:24 -0800 (PST) Received: from mail.crl.com (mail.crl.com [165.113.1.22]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA08759 for ; Tue, 3 Dec 1996 12:14:33 -0800 (PST) Received: from rocky.mt.sri.com by mail.crl.com with SMTP id AA09367 (5.65c/IDA-1.5 for ); Tue, 3 Dec 1996 12:15:13 -0800 Received: (from nate@localhost) by rocky.mt.sri.com (8.7.5/8.7.3) id NAA06474; Tue, 3 Dec 1996 13:07:00 -0700 (MST) Date: Tue, 3 Dec 1996 13:07:00 -0700 (MST) Message-Id: <199612032007.NAA06474@rocky.mt.sri.com> From: Nate Williams To: Steve Passe Cc: Erich Boleyn , smp@freebsd.org Subject: Re: APIC_IO stuff In-Reply-To: <199612031944.MAA00118@clem.systemsix.com> References: <199612031944.MAA00118@clem.systemsix.com> Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > I haven't had much time to try to solve the syscons problem, so I've > > been running SMP by remote login over ethernet. ... > resup and retry, the version you have was broken big-time. you should > be able to run 2 CPUS & APIC_IO without problem. If syscons is still > a problem you could try using pcvt instead. Kazu's changes should have fixed *all* of the syscons problems. If not then let us know! Nate From owner-freebsd-smp Tue Dec 3 12:21:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA09081 for smp-outgoing; Tue, 3 Dec 1996 12:21:46 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id MAA09074 for ; Tue, 3 Dec 1996 12:21:44 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id NAA00318; Tue, 3 Dec 1996 13:12:51 -0700 Message-Id: <199612032012.NAA00318@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Nate Williams cc: Erich Boleyn , smp@freebsd.org Subject: Re: APIC_IO stuff In-reply-to: Your message of "Tue, 03 Dec 1996 13:07:00 MST." <199612032007.NAA06474@rocky.mt.sri.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Dec 1996 13:12:51 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, would these changes have made it into the merge Peter did last nite: peter 96/12/03 03:42:14 sys - Imported sources Update of /home/smp/sys In directory freefall.freebsd.org:/f/peter/work/sys Revision/Branch: 1.1.1 Log Message: Import -current 961203 Status: Vendor Tag: CURRENT Release Tags: v961203 ... U sys/i386/isa/syscons.c -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Tue Dec 3 12:35:50 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA10233 for smp-outgoing; Tue, 3 Dec 1996 12:35:50 -0800 (PST) Received: from rocky.mt.sri.com (rocky.mt.sri.com [206.127.76.100]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA10226 for ; Tue, 3 Dec 1996 12:35:48 -0800 (PST) Received: (from nate@localhost) by rocky.mt.sri.com (8.7.5/8.7.3) id NAA06629; Tue, 3 Dec 1996 13:35:43 -0700 (MST) Date: Tue, 3 Dec 1996 13:35:43 -0700 (MST) Message-Id: <199612032035.NAA06629@rocky.mt.sri.com> From: Nate Williams To: Steve Passe Cc: Nate Williams , Erich Boleyn , smp@freebsd.org Subject: Re: APIC_IO stuff In-Reply-To: <199612032012.NAA00318@clem.systemsix.com> References: <199612032007.NAA06474@rocky.mt.sri.com> <199612032012.NAA00318@clem.systemsix.com> Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > would these changes have made it into the merge Peter did last nite: Yep, the changes were committed on Sunday, and Peter imported them on Monday. I just checked on freefall and they files in the SMP tree are the latest/greatest versions. :) Nate From owner-freebsd-smp Tue Dec 3 12:55:43 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA11175 for smp-outgoing; Tue, 3 Dec 1996 12:55:43 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA11168 for ; Tue, 3 Dec 1996 12:55:37 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id QAA14333; Tue, 3 Dec 1996 16:52:28 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612032052.QAA14333@bluenose.na.tuns.ca> Subject: Re: Great ! To: smp@csn.net (Steve Passe) Date: Tue, 3 Dec 1996 16:52:28 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612031906.MAA29775@clem.systemsix.com> from Steve Passe at "Dec 3, 96 12:06:22 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > This is great news for me - since 19:50 MET the SMP kernel is > > __UP__ and __RUNNING__ ! No more IDE-problems ! > > its even greater news to me, I was about to go out and buy an IDE > drive to try and figure out what the f*^& was happening! > > Important: for the record, do you have APIC_IO and IDE working? > > --- > > What was the reason ? > > it was *caused* by our attempts to autostart the 2nd CPU, as to the > *reason*, I'd consider giving up my right arm to know why... > I still have IDE problem :-(. For the latest smp kernel (supped this morning), if the second cpu is not launched, everything works just fine but as soon as the second cpu is activated, coredump shows up! On the other hand, if the system is booted from SCSI drive, it seems no problem. motherboard: Tomcat III smp kernel with option APIC_IO Jim From owner-freebsd-smp Tue Dec 3 13:09:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id NAA11898 for smp-outgoing; Tue, 3 Dec 1996 13:09:01 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id NAA11893 for ; Tue, 3 Dec 1996 13:08:57 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id PAA15610 for ; Tue, 3 Dec 1996 15:08:32 -0600 (CST) Message-Id: <199612032108.PAA15610@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 cc: freebsd-smp@freebsd.org Subject: Finished locks.. In-reply-to: Your message of Tue, 03 Dec 1996 00:54:40 -0600. <199612030654.AAA12221@friley216.res.iastate.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 03 Dec 1996 15:08:32 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk For what it's worth, I have completed a simple implementation of spin locks. (However, I'm sure someone will tell me different.. :) Either way, it was nice to learn a bit of assembly. If these are not sufficient, how can I improve upon them? I'd like to move some of the locking around, and they would help. In particular, I am planning to move the global lock out into the syscalls, and perhaps create a seperate lock for the run queues and such. Then from there, we can we can reduce the graininess a bit. :) Anyway, I'd just like to thank all of you for helping me out. Chris .text /*********************************************************************** * void LOCK() * ----------------- * All registers preserved */ .align 2,0x90 .globl _LOCK _LOCK: pushl %eax pushl %ecx pushl %edx movl $_smp_active, %eax cmpl $0, %eax je 4f 1: movl 16(%esp), %edx movl $1, %ecx movl $0, %eax 2: lock cmpxchg %ecx, (%edx) jne 3f jmp 4f 3: cmpl %ecx, (%edx) je 3b jmp 2b 4: popl %edx popl %ecx popl %eax ret /*********************************************************************** * int TRY_LOCK() * ----------------- * reg %eax == 1 if success */ .align 2,0x90 .globl _TRY_LOCK _TRY_LOCK: pushl %ecx pushl %edx movl $_smp_active, %eax cmpl $0, %eax je 1f movl 12(%esp), %edx movl $1, %ecx movl $0, %eax lock cmpxchg %ecx, (%edx) jne 1f movl $1, %eax jmp 2f 1: movl $0, %eax 2: popl %edx popl %ecx ret /*********************************************************************** * void UNLOCK() * ----------------- * All registers preserved */ .align 2,0x90 .globl _UNLOCK _UNLOCK: pushl %ecx movl 8(%esp), %ecx movl $0, (%ecx) popl %ecx ret From owner-freebsd-smp Tue Dec 3 14:39:29 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id OAA17162 for smp-outgoing; Tue, 3 Dec 1996 14:39:29 -0800 (PST) Received: from cs.utah.edu (cs.utah.edu [128.110.4.21]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id OAA17154 for ; Tue, 3 Dec 1996 14:39:27 -0800 (PST) Received: from fast.cs.utah.edu by cs.utah.edu (8.8.3/utah-2.21-cs) id PAA28898; Tue, 3 Dec 1996 15:39:20 -0700 (MST) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id PAA15211; Tue, 3 Dec 1996 15:39:19 -0700 Date: Tue, 3 Dec 1996 15:39:19 -0700 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199612032239.PAA15211@fast.cs.utah.edu> To: ccsanady@friley216.res.iastate.edu Subject: Re: Finished locks.. Cc: freebsd-smp@freebsd.org Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk There are a few problems with your code: 1) It can't be inlined by the compiler. Especially Unlock, which is trivial, should be inlined for performance. (With fine-grained locking, there are tens of thousands of them a second...if not more) 2) You are pessemistic about the register usage. If you use a gcc __asm__ macro, gcc will know if it can trash registers; this will save considerable time. (It can also arrange for the values to be loaded in advance, or whatever). By allowing the compiler to pick the registers whenever possible ("r"), it can better manage the register usage. I rewrote my recursive lock functions as macros, which is the reason I discovered a "feature" of gcc: if you pre-load eax with -1 (in param) it won't work if the code is inlined, as gcc trashed the IN parameter as it reused the variable; ie, %eax, an IN parameter, was also assigned as "r"). (This was for the three-register cmpxchg; it specified %eax as a register, even though it was already implicit). This was VERY annoying to debug. (I just had to look at the assembly output). Why gcc thinks it can re-use IN registers is beyond me. Kevin T. Van Maren University of Utah, CSL From owner-freebsd-smp Tue Dec 3 17:53:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id RAA28045 for smp-outgoing; Tue, 3 Dec 1996 17:53:01 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id RAA28037 for ; Tue, 3 Dec 1996 17:52:57 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id JAA08174; Wed, 4 Dec 1996 09:52:08 +0800 (WST) Message-Id: <199612040152.JAA08174@spinner.DIALix.COM> To: Nate Williams cc: Steve Passe , Erich Boleyn , smp@freebsd.org Subject: Re: APIC_IO stuff In-reply-to: Your message of "Tue, 03 Dec 1996 13:07:00 MST." <199612032007.NAA06474@rocky.mt.sri.com> Date: Wed, 04 Dec 1996 09:52:07 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Nate Williams wrote: > > > I haven't had much time to try to solve the syscons problem, so I've > > > been running SMP by remote login over ethernet. > ... > > resup and retry, the version you have was broken big-time. you should > > be able to run 2 CPUS & APIC_IO without problem. If syscons is still > > a problem you could try using pcvt instead. > > Kazu's changes should have fixed *all* of the syscons problems. If not > then let us know! I suspected so, I merged -current in again last night after seeing that the dust seemed to have died down over syscons.. > Nate Cheers, -Peter From owner-freebsd-smp Tue Dec 3 18:45:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA00614 for smp-outgoing; Tue, 3 Dec 1996 18:45:01 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA00600 for freebsd-smp; Tue, 3 Dec 1996 18:44:59 -0800 (PST) Date: Tue, 3 Dec 1996 18:44:59 -0800 (PST) From: Peter Wemm Message-Id: <199612040244.SAA00600@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/kern init_smp.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 18:44:58 Modified: kern init_smp.c Log: Set default smp.invldebug to 6 if SMP_INVLTLB is defined. Revision Changes Path 1.42 +5 -2 sys/kern/init_smp.c From owner-freebsd-smp Tue Dec 3 18:47:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA00740 for smp-outgoing; Tue, 3 Dec 1996 18:47:01 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA00731 for freebsd-smp; Tue, 3 Dec 1996 18:47:00 -0800 (PST) Date: Tue, 3 Dec 1996 18:47:00 -0800 (PST) From: Peter Wemm Message-Id: <199612040247.SAA00731@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/isa vector.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 18:47:00 Modified: i386/isa vector.s Log: Fix IPI_INTR handler code. It was not setting %ds with the kernel selector, so if it took an IPI while in user mode, it got a GPF when trying to access kernel data. (%cs is set by the hardware) Revision Changes Path 1.30 +16 -2 sys/i386/isa/vector.s From owner-freebsd-smp Tue Dec 3 18:52:01 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA01016 for smp-outgoing; Tue, 3 Dec 1996 18:52:01 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA00999 for freebsd-smp; Tue, 3 Dec 1996 18:51:58 -0800 (PST) Date: Tue, 3 Dec 1996 18:51:58 -0800 (PST) From: Peter Wemm Message-Id: <199612040251.SAA00999@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/include cpufunc.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 18:51:57 Modified: i386/include cpufunc.h Log: When using SMP_INVLTLB, dont inline invltlb() and invlpg() functions since they grow a fair bit and become expensive to inline.. Not to mention difficult since it causes cpufunc.h to be dependent on #including half the apic systems... Revision Changes Path 1.13 +14 -3 sys/i386/include/cpufunc.h From owner-freebsd-smp Tue Dec 3 18:55:07 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA01114 for smp-outgoing; Tue, 3 Dec 1996 18:55:07 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA01107 for freebsd-smp; Tue, 3 Dec 1996 18:55:05 -0800 (PST) Date: Tue, 3 Dec 1996 18:55:05 -0800 (PST) From: Peter Wemm Message-Id: <199612040255.SAA01107@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mp_machdep.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 18:55:05 Modified: i386/i386 mp_machdep.c Log: Improve the rudimentry support for doing an invltlb via IPI 27.. Revision Changes Path 1.27 +54 -21 sys/i386/i386/mp_machdep.c From owner-freebsd-smp Tue Dec 3 18:56:16 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA01216 for smp-outgoing; Tue, 3 Dec 1996 18:56:16 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id SAA01209 for freebsd-smp; Tue, 3 Dec 1996 18:56:15 -0800 (PST) Date: Tue, 3 Dec 1996 18:56:15 -0800 (PST) From: Peter Wemm Message-Id: <199612040256.SAA01209@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/conf options.i386 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 18:56:15 Modified: i386/conf options.i386 Log: list SMP_INVLTLB as "should be working" Revision Changes Path 1.15 +3 -1 sys/i386/conf/options.i386 From owner-freebsd-smp Tue Dec 3 19:02:19 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA01454 for smp-outgoing; Tue, 3 Dec 1996 19:02:19 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA01416; Tue, 3 Dec 1996 19:02:10 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id LAA00443; Wed, 4 Dec 1996 11:02:06 +0800 (WST) Message-Id: <199612040302.LAA00443@spinner.DIALix.COM> To: Peter Wemm cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/isa vector.s In-reply-to: Your message of "Tue, 03 Dec 1996 18:47:00 PST." <199612040247.SAA00731@freefall.freebsd.org> Date: Wed, 04 Dec 1996 11:02:06 +0800 From: Peter Wemm Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Peter Wemm wrote: > peter 96/12/03 18:47:00 > > Modified: i386/isa vector.s > Log: > Fix IPI_INTR handler code. It was not setting %ds with the kernel > selector, so if it took an IPI while in user mode, it got a GPF when > trying to access kernel data. (%cs is set by the hardware) I'm not sure why this used to work before.. It should never have.. I somehow suspect that this might be somehow related to the SMP_AUTOSTART code that is now disabled. I suspect all the tests were done while it was unconditionally active. This might be a clue as to what was going wrong with the code.. It looks like %ds was always set to the kernel data segment selector, even while in user mode on both cpu's. This would have been rather bad if so.. I have no idea how this could have been caused though. Cheers, -Peter From owner-freebsd-smp Tue Dec 3 19:04:39 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA01580 for smp-outgoing; Tue, 3 Dec 1996 19:04:39 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA01575; Tue, 3 Dec 1996 19:04:34 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id LAA00465; Wed, 4 Dec 1996 11:04:29 +0800 (WST) Message-Id: <199612040304.LAA00465@spinner.DIALix.COM> To: Peter Wemm cc: freebsd-smp@freefall.freebsd.org Subject: Re: cvs commit: sys/i386/conf options.i386 In-reply-to: Your message of "Tue, 03 Dec 1996 18:56:15 PST." <199612040256.SAA01209@freefall.freebsd.org> Date: Wed, 04 Dec 1996 11:04:29 +0800 From: Peter Wemm Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Peter Wemm wrote: > peter 96/12/03 18:56:15 > > Modified: i386/conf options.i386 > Log: > list SMP_INVLTLB as "should be working" Incidently, I'd appreciate hearing if this works for other people as well. If this works as I suspect it does, it will dramatically reduce the chances of filesystem and process memory corruption due to page stealing and should be a major reliability improvement. Cheers, -Peter From owner-freebsd-smp Tue Dec 3 19:06:13 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA01680 for smp-outgoing; Tue, 3 Dec 1996 19:06:13 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA01672 for freebsd-smp; Tue, 3 Dec 1996 19:06:12 -0800 (PST) Date: Tue, 3 Dec 1996 19:06:12 -0800 (PST) From: Peter Wemm Message-Id: <199612040306.TAA01672@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mp_machdep.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/03 19:06:11 Modified: i386/i386 mp_machdep.c Log: oops, the invltlb debug code was dependent on SERIAL_DEBUG being enabled. Revision Changes Path 1.28 +6 -1 sys/i386/i386/mp_machdep.c From owner-freebsd-smp Tue Dec 3 19:25:42 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA02342 for smp-outgoing; Tue, 3 Dec 1996 19:25:42 -0800 (PST) Received: from base486.synet.net (imdave@DIAL10.SYNET.NET [168.113.1.12]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA02328 for ; Tue, 3 Dec 1996 19:25:37 -0800 (PST) Received: (from imdave@localhost) by base486.synet.net (8.8.4/8.8.4) id VAA13549; Tue, 3 Dec 1996 21:25:20 -0600 (CST) Date: Tue, 3 Dec 1996 21:25:20 -0600 (CST) From: Dave Bodenstab Message-Id: <199612040325.VAA13549@base486.synet.net> To: ccsanady@friley216.res.iastate.edu Subject: Re: Finished locks.. Cc: freebsd-smp@freebsd.org Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > For what it's worth, I have completed a simple implementation of spin > locks. (However, I'm sure someone will tell me different.. :) Either > way, it was nice to learn a bit of assembly. > > If these are not sufficient, how can I improve upon them? I'd like to > move some of the locking around, and they would help. In particular, I > am planning to move the global lock out into the syscalls, and perhaps > create a separate lock for the run queues and such. Then from there, we > can we can reduce the graininess a bit. :) Hi, Just thought I'd try to contribute a bit... maybe next year I'll get a MP board and try to dig back into kernel hacking... Back around 1988-1991 (or sometime a while ago) I worked on the SVR3 kernel for AT&T's 3B2 non-symmetric multiprocessor system. It was non-symmetric because only the `main' processor had access to the I/O bus. We initially modified the scheduler to only run user-level code on the `co-processors'. Then, bit by bit, we began to move the locking deeper into the kernel. As for the locks, we found it very useful during our development to have debugging versions of the locks. We used in-line assembler -- AT&T's assembler allowed this, and so does GNU's for FreeBSD. The debugging versions of the spin locks maintained simple counts and statistics for each lock (which we gave a global name): how many times hit how many times blocked number of spins when blocked the instruction pointer of who owns the lock the pid of who owns the lock etc. We then wrote a simple utility that used nlist to dump out the counts of a running kernel or the crash dump image. (The utility had the names of the locks built-in.) It was *very* useful to actually measure -- although crudely -- what was going on. You might want to consider doing this for your spin-locks. Good luck! I'm looking forward to running a SMP kernel someday! Dave Bodenstab imdave@synet.net PS. Another *very* useful thing is to implement a `trace' package. We sprinkled calls that looked like: monitor( char id, long arg1, long arg2, long arg3, long arg4 ); in key areas of the kernel: interrupt entry and exit scheduler entry and exit system call entry and exit trap entry and exit signal delivery For each call, the argn's were pertinent information relating to that particular kernel routine; for the scheduler we'd save the old and new pid's, and perhaps some of the scheduling criteria, for instance. A production kernel had the calls #ifdef'ed out. The monitor package stored the arguments into a large (2048+ slots), circular buffer, adding the current lbolt value and the return address of the caller. Several of the `id' characters were pre-defined, for instance we used 'P' for scheduler entry and 'p' for scheduler exit. For interrupt routines, the code would check to see if the last entry stored was for the same interrupt routine, and if so, just bump a counter -- this prevented the clock interrupts from flooding the buffer and wiping out all the other information in cases where the system locked up. Another utility was written to dump the contents of the buffer (again either from a running system or a crash image) and format it properly -- we knew what each `id' character was used for and what information each of the argn's was. What this gives you is a real-time trace of key kernel activity. It's much like sticking printf's into the kernel, but *much* more practical. There is much less chance that the additional debugging code will perturb the system as printf's will. This trace package proved even more useful when we had multiple processors! Any time one had to investigate a problem, one could sprinkle additional calls throughout the code as one zeros in on the problem. Post-mortem analysis was also tremendously improved. From owner-freebsd-smp Tue Dec 3 19:49:55 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id TAA03439 for smp-outgoing; Tue, 3 Dec 1996 19:49:55 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id TAA03434 for ; Tue, 3 Dec 1996 19:49:51 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id XAA15194; Tue, 3 Dec 1996 23:46:45 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612040346.XAA15194@bluenose.na.tuns.ca> Subject: Re: cvs commit: sys/i386/conf options.i386 To: peter@spinner.dialix.com (Peter Wemm) Date: Tue, 3 Dec 1996 23:46:44 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612040304.LAA00465@spinner.DIALix.COM> from Peter Wemm at "Dec 4, 96 11:04:29 am" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > Modified: i386/conf options.i386 > > Log: > > list SMP_INVLTLB as "should be working" > > Incidently, I'd appreciate hearing if this works for other people > as well. If this works as I suspect it does, it will dramatically > reduce the chances of filesystem and process memory corruption due > to page stealing and should be a major reliability improvement. > No more coredump for IDE with options SMP_INVLTLB. Great Work! Jim From owner-freebsd-smp Tue Dec 3 21:49:23 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id VAA07692 for smp-outgoing; Tue, 3 Dec 1996 21:49:23 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id VAA07687 for ; Tue, 3 Dec 1996 21:49:20 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVAGv-0000q5-00; Tue, 3 Dec 1996 21:53:29 -0800 To: smp@freebsd.org Subject: Crashing on activating other CPUs Date: Tue, 03 Dec 1996 21:53:29 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I synced up my tree here, and tried it out with and without APIC_IO (and with/without SMP_INVLTLB). The syscons problems are fixed (yayy!), and it runs well as long as I don't try to start any other CPUs (via "sysctl -w kern.smp_active=2"). If I do the above command, a big flurry of: Fatal trap 12: page fault while in kernel mode ...and full dump messages run by before it freezes solid (and a reset is necessary). No variation on SMP_INVLTLB or APIC_IO seems to change the above behavior. The last messages say: cpunumber = 2 fault virtual address = 0xa0 fault code = supervisor read, page not present instruction pointer = 0x8:0xf01150d9 stack pointer = 0x10:0xf0223c12 frame pointer = 0x10:0xf0223c22 ... current precess = Idle interrupt mask = tty panic (cpu#2): page fault boot() called on cpu#2 I'll sniff around to see if I can find where it is in the code tomorrow (my wife is saying I must come to bed or I won't live to see the morning ;-). I suspect it might be a greater than 2 CPUs problem (question: should I be saying "kern.smp_active=4" on my 4 CPU test box??). -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Wed Dec 4 03:04:35 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id DAA12160 for smp-outgoing; Wed, 4 Dec 1996 03:04:35 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id DAA12141 for ; Wed, 4 Dec 1996 03:04:22 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id TAA04123; Wed, 4 Dec 1996 19:03:39 +0800 (WST) Message-Id: <199612041103.TAA04123@spinner.DIALix.COM> To: "J.M. Chuang" cc: smp@freebsd.org Subject: Re: cvs commit: sys/i386/conf options.i386 In-reply-to: Your message of "Tue, 03 Dec 1996 23:46:44 -0400." <199612040346.XAA15194@bluenose.na.tuns.ca> Date: Wed, 04 Dec 1996 19:03:39 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk "J.M. Chuang" wrote: > > > Modified: i386/conf options.i386 > > > Log: > > > list SMP_INVLTLB as "should be working" > > > > Incidently, I'd appreciate hearing if this works for other people > > as well. If this works as I suspect it does, it will dramatically > > reduce the chances of filesystem and process memory corruption due > > to page stealing and should be a major reliability improvement. > > > No more coredump for IDE with options SMP_INVLTLB. Great Work! Hey, wow! This is an unexpected bonus! I never expected this to be a side effect, but now that I think about it, it makes sense. The SCSI system generally uses bus-mastering to send the data directly to the destination physical address, while the IDE driver maps the target page into KVM and does PIO to write it. And since it's changing mappings, it would be very vulnerable to stale tlb data. The remaining problem is that we are currently doing way too many global invltlb() calls. John Dyson has already indicated that we can be a lot more selective about which ones we propagate to other cpus. Right now, I can see the IPI's (which show up as "irq0" interrupts on systat -vmstat) peaking in the 2000+ per second range which is rather excessive.. This is a performance problem, not a reliability one. > Jim Cheers, -Peter From owner-freebsd-smp Wed Dec 4 04:35:50 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id EAA15720 for smp-outgoing; Wed, 4 Dec 1996 04:35:50 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id EAA15715 for ; Wed, 4 Dec 1996 04:35:46 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id NAA02260 for ; Wed, 4 Dec 1996 13:35:42 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id NAA06405 for ; Wed, 4 Dec 1996 13:35:41 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id NAA00889 for ; Wed, 4 Dec 1996 13:35:40 +0100 (MET) Message-Id: <199612041235.NAA00889@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Tue, 03 Dec 1996 21:53:29 MET." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 13:35:40 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >...and full dump messages run by before it freezes solid (and a reset is >necessary). No variation on SMP_INVLTLB or APIC_IO seems to change the >above behavior. Our 4 CPU Netserver works without SMP_INVLTLB and APIC_IO (the latter has never worked). With SMP_INVLTLB, I get something similar the moment I do "sysctl -w kern.smp_active=2". SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... cpunumber = 0 instruction pointer = 0x8:0xf01b632d stack pointer = 0x10:0xefbfffa4 frame pointer = 0x10:0xefbfd5e8 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 149 (sshd@386freebsd2) interrupt mask = panic (cpu#0): unknown/reserved trap boot() called on cpu#0 syncing disks... 5 5 2 done Automatic reboot in 15 seconds - press a key on the console to abort Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Wed Dec 4 05:18:04 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id FAA29018 for smp-outgoing; Wed, 4 Dec 1996 05:18:04 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id FAA28979 for ; Wed, 4 Dec 1996 05:17:59 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id VAA00740; Wed, 4 Dec 1996 21:16:39 +0800 (WST) Message-Id: <199612041316.VAA00740@spinner.DIALix.COM> To: Terje Normann Marthinussen cc: smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 13:35:40 +0100." <199612041235.NAA00889@slibo.cc.uit.no> Date: Wed, 04 Dec 1996 21:16:39 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Terje Normann Marthinussen wrote: > >...and full dump messages run by before it freezes solid (and a reset is > >necessary). No variation on SMP_INVLTLB or APIC_IO seems to change the > >above behavior. > > Our 4 CPU Netserver works without SMP_INVLTLB and APIC_IO (the latter > has never worked). With SMP_INVLTLB, I get something similar the > moment I do "sysctl -w kern.smp_active=2". > > SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... > cpunumber = 0 > instruction pointer = 0x8:0xf01b632d ^^^^^^^^^^^^^^^^ > stack pointer = 0x10:0xefbfffa4 > frame pointer = 0x10:0xefbfd5e8 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, IOPL = 0 > current process = 149 (sshd@386freebsd2) > interrupt mask = > panic (cpu#0): unknown/reserved trap > boot() called on cpu#0 > > syncing disks... 5 5 2 done > Automatic reboot in 15 seconds - press a key on the console to abort > > > Terje Marthinussen > terjem@cc.uit.no Hmm, you have all 4 cpu's enabled? Steve sounds like he's figured out why the >2 cpu startup code (presently disabled) causes problems. We do not support >2 cpu's at the moment. ("do not support" == "known to not work" in this case). We are close, but still not quite there. Can you please try two things.. First, where is 0xf01b632d ? Can you please do this: # nm /kernel.whatever | sort | more and search for the the 3 surrounding function names? Second, can you please compile with "options DDB" and when it does this, do a "trace" command and record the output. Cheers, -Peter From owner-freebsd-smp Wed Dec 4 06:13:21 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA28582 for smp-outgoing; Wed, 4 Dec 1996 06:13:21 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA28572 for freebsd-smp; Wed, 4 Dec 1996 06:13:18 -0800 (PST) Date: Wed, 4 Dec 1996 06:13:18 -0800 (PST) From: Peter Wemm Message-Id: <199612041413.GAA28572@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 trap.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/04 06:13:17 Modified: i386/i386 trap.c Log: print fatal trap number. This way, when we get "unknown/unrecognised trap" messages, we might have more of an idea what we're up against. Revision Changes Path 1.14 +1 -0 sys/i386/i386/trap.c From owner-freebsd-smp Wed Dec 4 06:15:10 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA28978 for smp-outgoing; Wed, 4 Dec 1996 06:15:10 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA28963 for freebsd-smp; Wed, 4 Dec 1996 06:15:08 -0800 (PST) Date: Wed, 4 Dec 1996 06:15:08 -0800 (PST) From: Peter Wemm Message-Id: <199612041415.GAA28963@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/isa isa.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/04 06:15:07 Modified: i386/isa isa.c Log: deal with irq > 19 in register_intr. Revision Changes Path 1.12 +4 -1 sys/i386/isa/isa.c From owner-freebsd-smp Wed Dec 4 06:19:40 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA01107 for smp-outgoing; Wed, 4 Dec 1996 06:19:40 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA01012 for ; Wed, 4 Dec 1996 06:19:30 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id PAA04404; Wed, 4 Dec 1996 15:18:53 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id PAA07062; Wed, 4 Dec 1996 15:18:52 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id PAA03009; Wed, 4 Dec 1996 15:18:51 +0100 (MET) Message-Id: <199612041418.PAA03009@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Peter Wemm cc: smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 21:16:39 MET." <199612041316.VAA00740@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Wed, 04 Dec 1996 15:18:51 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >Terje Normann Marthinussen wrote: >> >...and full dump messages run by before it freezes solid (and a reset is >> >necessary). No variation on SMP_INVLTLB or APIC_IO seems to change the >> >above behavior. >> >> Our 4 CPU Netserver works without SMP_INVLTLB and APIC_IO (the latter >> has never worked). With SMP_INVLTLB, I get something similar the >> moment I do "sysctl -w kern.smp_active=2". >> >> SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... >> cpunumber = 0 >> instruction pointer = 0x8:0xf01b632d > ^^^^^^^^^^^^^^^^ >> stack pointer = 0x10:0xefbfffa4 >> frame pointer = 0x10:0xefbfd5e8 >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, def32 1, gran 1 >> processor eflags = interrupt enabled, IOPL = 0 >> current process = 149 (sshd@386freebsd2) >> interrupt mask = >> panic (cpu#0): unknown/reserved trap >> boot() called on cpu#0 >> >> syncing disks... 5 5 2 done >> Automatic reboot in 15 seconds - press a key on the console to abort >> >> >> Terje Marthinussen >> terjem@cc.uit.no > >Hmm, you have all 4 cpu's enabled? Steve sounds like he's figured out >why the >2 cpu startup code (presently disabled) causes problems. We >do not support >2 cpu's at the moment. ("do not support" == "known to >not work" in this case). We are close, but still not quite there. No, CPU 3 and 4 fails to start, so I've put: mp_naps=1; into mp_machdep.c to avoid trying to start them. > >Can you please try two things.. > >First, where is 0xf01b632d ? Can you please do this: ># nm /kernel.whatever | sort | more >and search for the the 3 surrounding function names? Argh.. don't have that kernel anymore. However, after recompiling with DDB, I got: cpunumber = 0 instruction pointer = 0x8:0xf010d5ad stack pointer = 0x10:0xefbfff68 frame pointer = 0x10:0xefbfff6c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 6 (cpuidle1) interrupt mask = kernel: type 29 trap, code=0 Stopped at _smp_idleloop+0x3d: andl $0xf000000,%eax db> trace _smp_idleloop(f40c14d7,f010d369,1,f01e8294,f01ee958) at _smp_idleloop+0x3d _smp_kickoff(0) at _smp_kickoff+0x97 _main(efbfffb8,f011cec0,f01ba140,8000000,5,0,efbffff4,f01bd450,f01bcbe3,80000011 ,30,22ff00,233000) at _main+0x92 >From /kernel f010d290 F init_smp.o f010d290 t _sysctl___kern_smp_active f010d2bc t ___set_sysctl__kern_sym_sysctl___kern_smp_active f010d2c0 t _sysctl___kern_idle_debug f010d2e8 t ___set_sysctl__kern_sym_sysctl___kern_idle_debug f010d2ec t _sysctl___kern_invldebug f010d314 t ___set_sysctl__kern_sym_sysctl___kern_invldebug f010d318 t _sysctl___kern_ignore_idleprocs f010d348 t ___set_sysctl__kern_sym_sysctl___kern_ignore_idleprocs f010d34c t ___set_sysinit_set_sym_smpkick_sys_init f010d388 t _smp_kickoff f010d488 T _secondary_main f010d570 t _smp_idleloop f010d7d0 F init_sysent.o f010d7d0 F init_sysvec.o f010d7e0 F kern_acct.o f010d7e0 t _sysctl___kern_acct_suspend f010d80c t ___set_sysctl__kern_sym_sysctl___kern_acct_suspend f010d810 t _sysctl___kern_acct_resume Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Wed Dec 4 06:20:17 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA01446 for smp-outgoing; Wed, 4 Dec 1996 06:20:17 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA01432 for freebsd-smp; Wed, 4 Dec 1996 06:20:15 -0800 (PST) Date: Wed, 4 Dec 1996 06:20:15 -0800 (PST) From: Peter Wemm Message-Id: <199612041420.GAA01432@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mp_machdep.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/04 06:20:15 Modified: i386/i386 mp_machdep.c Log: "borrow" the pci_ihandler_attach() code and adapt it for the ipi routines. I have a modified version of config(8) which has 4 slots reserved for the ipi interrupt stats. This allows "vmstat -i" and "systat -vmstat 1" to be aware of the ipi counts when SMP_INVLTLB is in use. If the user is not running a "smp aware" config(8), it'll still append the counts to clk0 just like it did before. Revision Changes Path 1.29 +62 -9 sys/i386/i386/mp_machdep.c From owner-freebsd-smp Wed Dec 4 06:28:18 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA03560 for smp-outgoing; Wed, 4 Dec 1996 06:28:18 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA03535 for ; Wed, 4 Dec 1996 06:28:12 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id WAA01519; Wed, 4 Dec 1996 22:27:25 +0800 (WST) Message-Id: <199612041427.WAA01519@spinner.DIALix.COM> To: Terje Normann Marthinussen cc: smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 15:18:51 +0100." <199612041418.PAA03009@slibo.cc.uit.no> Date: Wed, 04 Dec 1996 22:27:25 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Terje Normann Marthinussen wrote: > However, after recompiling with DDB, I got: > cpunumber = 0 > instruction pointer = 0x8:0xf010d5ad > stack pointer = 0x10:0xefbfff68 > frame pointer = 0x10:0xefbfff6c > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, IOPL = 0 > current process = 6 (cpuidle1) > interrupt mask = > kernel: type 29 trap, code=0 > Stopped at _smp_idleloop+0x3d: andl $0xf000000,%eax > db> trace > _smp_idleloop(f40c14d7,f010d369,1,f01e8294,f01ee958) at _smp_idleloop+0x3d > _smp_kickoff(0) at _smp_kickoff+0x97 > _main(efbfffb8,f011cec0,f01ba140,8000000,5,0,efbffff4,f01bd450,f01bcbe3,80000 011 > ,30,22ff00,233000) at _main+0x92 Hmm!! I do not understand this at all... How can an 'and' between an immediate and a register generate a trap??? Are you using a P6? Perhaps they have an extra trap or two defined for some new condition? Cheers, -Peter From owner-freebsd-smp Wed Dec 4 06:41:37 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA06382 for smp-outgoing; Wed, 4 Dec 1996 06:41:37 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA06357 for ; Wed, 4 Dec 1996 06:41:32 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id WAA00398 for ; Wed, 4 Dec 1996 22:41:28 +0800 (WST) Message-Id: <199612041441.WAA00398@spinner.DIALix.COM> To: smp@freebsd.org Subject: Re: cvs commit: sys/i386/i386 mp_machdep.c In-reply-to: Your message of "Wed, 04 Dec 1996 06:20:15 PST." <199612041420.GAA01432@freefall.freebsd.org> Date: Wed, 04 Dec 1996 22:41:26 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Peter Wemm wrote: > "borrow" the pci_ihandler_attach() code and adapt it for the ipi > routines. I have a modified version of config(8) which has 4 slots > reserved for the ipi interrupt stats. This allows "vmstat -i" and > "systat -vmstat 1" to be aware of the ipi counts when SMP_INVLTLB > is in use. If the user is not running a "smp aware" config(8), it'll > still append the counts to clk0 just like it did before. Incidently, directly after booting and logging in: interrupt total rate clk0 irq2 8568 129 rtc0 irq8 8498 128 ipi irq27 33284 504 sc0 irq1 280 4 sio0 irq4 9 0 lpt0 irq7 1 0 ed0 irq5 134 2 stray irq13 1 0 Total 50775 769 .. and then doing a 'cvs -q update -d -P -A' on the smp tree: interrupt total rate clk0 irq2 23125 138 rtc0 irq8 21474 128 ipi irq27 53146 318 sc0 irq1 415 2 sio0 irq4 9 0 lpt0 irq7 1 0 ed0 irq5 153 0 stray irq13 1 0 Total 98324 588 Note that my eisa disk irq's show up under clk0. (Sigh!) Still, that's a hell of a load of invltlb() calls... Cheers, -Peter From owner-freebsd-smp Wed Dec 4 06:46:34 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA07907 for smp-outgoing; Wed, 4 Dec 1996 06:46:34 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id GAA07892 for ; Wed, 4 Dec 1996 06:46:31 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id WAA00462 for ; Wed, 4 Dec 1996 22:46:18 +0800 (WST) Message-Id: <199612041446.WAA00462@spinner.DIALix.COM> To: smp@freebsd.org Subject: modified tools outside of kernel source tree Date: Wed, 04 Dec 1996 22:46:17 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I have a tweaked version of config that knows how to reserve device_id slots for the ipi interrupts. I figured the best thing to do with it in the short term was to import a version of config under a "tools/config" directory in the smp tree and apply my tweaks there. These changes are not essential at this point, but are useful to have. Once it's settled down a bit, we can move the changes to -current so we don't have to have two seperate versions of config. The "smp aware" changes do not affect operation on the standard kernel and build tree. I imagine Steve would also like to put his mptable.c code under the tools directory too. Does this sound OK? -Peter From owner-freebsd-smp Wed Dec 4 06:56:27 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id GAA10470 for smp-outgoing; Wed, 4 Dec 1996 06:56:27 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id GAA10459 for ; Wed, 4 Dec 1996 06:56:23 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVInb-0001v3-00; Wed, 4 Dec 1996 06:59:47 -0800 To: Peter Wemm cc: Terje.N.Marthinussen@cc.uit.no, smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 22:27:25 +0800." <199612041427.WAA01519@spinner.DIALix.COM> Date: Wed, 04 Dec 1996 06:59:47 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Peter Wemm writes: > Terje Normann Marthinussen wrote: > > However, after recompiling with DDB, I got: > > cpunumber = 0 > > instruction pointer = 0x8:0xf010d5ad ... > > kernel: type 29 trap, code=0 > > Stopped at _smp_idleloop+0x3d: andl $0xf000000,%eax > > db> trace > > _smp_idleloop(f40c14d7,f010d369,1,f01e8294,f01ee958) at _smp_idleloop+0x3d > > _smp_kickoff(0) at _smp_kickoff+0x97 > > _main(efbfffb8,f011cec0,f01ba140,8000000,5,0,efbffff4,f01bd450,f01bcbe3,80000 > 011 > > ,30,22ff00,233000) at _main+0x92 > > Hmm!! I do not understand this at all... How can an 'and' between an > immediate and a register generate a trap??? I'm going to look at this on my machine as soon as I get a few minutes to rub together as well, maybe we can track it down. > Are you using a P6? Perhaps they have an extra trap or two defined for > some new condition? (correct me if I'm wrong, Terje, but...) He's using an XXPRESS-bus machine, an HP NetServer, which is Pentium-based, unless HP has some Pentium Pro upgrade option I don't know about. I helped several people debug these boxes for Linux-SMP. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Wed Dec 4 07:27:41 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA14356 for smp-outgoing; Wed, 4 Dec 1996 07:27:41 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id HAA14334 for ; Wed, 4 Dec 1996 07:27:34 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id QAA05865; Wed, 4 Dec 1996 16:27:15 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id QAA07628; Wed, 4 Dec 1996 16:27:14 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id QAA04593; Wed, 4 Dec 1996 16:27:13 +0100 (MET) Message-Id: <199612041527.QAA04593@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Peter Wemm cc: smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 22:27:25 MET." <199612041427.WAA01519@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 16:27:12 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >Terje Normann Marthinussen wrote: >> However, after recompiling with DDB, I got: >> cpunumber = 0 >> instruction pointer = 0x8:0xf010d5ad >> stack pointer = 0x10:0xefbfff68 >> frame pointer = 0x10:0xefbfff6c >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, def32 1, gran 1 >> processor eflags = interrupt enabled, IOPL = 0 >> current process = 6 (cpuidle1) >> interrupt mask = >> kernel: type 29 trap, code=0 >> Stopped at _smp_idleloop+0x3d: andl $0xf000000,%eax >> db> trace >> _smp_idleloop(f40c14d7,f010d369,1,f01e8294,f01ee958) at _smp_idleloop+0x3d >> _smp_kickoff(0) at _smp_kickoff+0x97 >> _main(efbfffb8,f011cec0,f01ba140,8000000,5,0,efbffff4,f01bd450,f01bcbe3,80000 > 011 >> ,30,22ff00,233000) at _main+0x92 > >Hmm!! I do not understand this at all... How can an 'and' between an >immediate and a register generate a trap??? > >Are you using a P6? Perhaps they have an extra trap or two defined for >some new condition? No, it is a 4 CPU 133 MHZ P5 system. Just for the record, I did a continue in DDB just after this. Now I wonder, what happens when you get into DDB, do you get the code that generated the trap, or do get the code that cpu0 was at when you got the trap? Tried after a reboot: cpunumber = 0 instruction pointer = 0x8:0xf010d65b stack pointer = 0x10:0xefbfff68 frame pointer = 0x10:0xefbfff6c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 6 (cpuidle1) interrupt mask = kernel: type 29 trap, code=0 Stopped at _smp_idleloop+0xeb: cmpl $0,_whichrtqs db> trace _smp_idleloop(f40c14d7,f010d369,1,f01e8294,f01ee958) at _smp_idleloop+0xeb _smp_kickoff(0) at _smp_kickoff+0x97 _main(efbfffb8,f011cec0,f01ba140,8000000,5,0,efbffff4,f01bd450,f01bcbe3,80000011 ,30,22ff00,233000) at _main+0x92 begin() at begin+0x44 db> c cpunumber = 1 instruction pointer = 0x8:0xf010d5ad stack pointer = 0x10:0xefbfff68 frame pointer = 0x10:0xefbfff6c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 6 (cpuidle1) interrupt mask = kernel: type 29 trap, code=0 Stopped at _smp_idleloop+0x3d: andl $0xf000000,%eax db> trace _smp_idleloop(f40c14d7,f010d369,1,f01e8294,f01ee958) at _smp_idleloop+0x3d _smp_kickoff(0) at _smp_kickoff+0x97 _main(efbfffb8,f011cec0,f01ba140,8000000,5,0,efbffff4,f01bd450,f01bcbe3,80000011 ,30,22ff00,233000) at _main+0x92 begin() at begin+0x44 And then yet another reboot: SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... cpunumber = 0 instruction pointer = 0x8:0xf010d5ad stack pointer = 0x10:0xefbfff68 frame pointer = 0x10:0xefbfff6c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 5 (cpuidle0) interrupt mask = kernel: type 29 trap, code=0 Stopped at _smp_idleloop+0x3d: andl $0xf000000,%eax db> c cpunumber = 1 instruction pointer = 0x8:0xf01bf24d stack pointer = 0x10:0xefbfffa4 frame pointer = 0x10:0xefbfd5f4 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, IOPL = 0 current process = 149 (sshd@386freebsd2) interrupt mask = kernel: type 29 trap, code=0 Stopped at _MPgetlock+0x5d: cmpl $-0x1,0(%edx) db> trace _MPgetlock(efbfd644,efbfd624,0,efbfd698,5) at _MPgetlock+0x5d Does this mean that there was a trap generated by both instructions, or that one of them made it? did continue again, and it got up and running with 2 cpus. After this, going from 2->1->2 cpus works fine. Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Wed Dec 4 07:29:46 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA14619 for smp-outgoing; Wed, 4 Dec 1996 07:29:46 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id HAA14605 for ; Wed, 4 Dec 1996 07:29:42 -0800 (PST) Received: from critter.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0vVIYx-0003wHC; Wed, 4 Dec 96 06:44 PST Received: from critter.tfs.com (localhost [127.0.0.1]) by critter.tfs.com (8.8.2/8.8.2) with ESMTP id JAA01938; Wed, 4 Dec 1996 09:33:34 +0100 (MET) To: Chris Csanady cc: freebsd-smp@freebsd.org Subject: Re: Finished locks.. In-reply-to: Your message of "Tue, 03 Dec 1996 15:08:32 CST." <199612032108.PAA15610@friley216.res.iastate.edu> Date: Wed, 04 Dec 1996 09:33:34 +0100 Message-ID: <1936.849688414@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk In message <199612032108.PAA15610@friley216.res.iastate.edu>, Chris Csanady wri tes: >For what it's worth, I have completed a simple implementation of spin >locks. (However, I'm sure someone will tell me different.. :) Either >way, it was nice to learn a bit of assembly. > >If these are not sufficient, how can I improve upon them? I'd like to >move some of the locking around, and they would help. In particular, I >am planning to move the global lock out into the syscalls, and perhaps >create a seperate lock for the run queues and such. Then from there, we >can we can reduce the graininess a bit. :) > >Anyway, I'd just like to thank all of you for helping me out. > >Chris > > > > .text > >/*********************************************************************** > * void LOCK() > * ----------------- > * All registers preserved > */ > > .align 2,0x90 > .globl _LOCK >_LOCK: > pushl %eax > pushl %ecx > pushl %edx > movl $_smp_active, %eax > cmpl $0, %eax > je 4f >1: movl 16(%esp), %edx > movl $1, %ecx > movl $0, %eax >2: lock > cmpxchg %ecx, (%edx) -> jne 3f -> jmp 4f + je 4f >3: cmpl %ecx, (%edx) > je 3b I'm sure you don't mean the above jump. > jmp 2b >4: popl %edx > popl %ecx > popl %eax > ret -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Wed Dec 4 07:31:29 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id HAA14799 for smp-outgoing; Wed, 4 Dec 1996 07:31:29 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id HAA14784 for ; Wed, 4 Dec 1996 07:31:25 -0800 (PST) Received: from critter.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0vVIZ0-0003wLC; Wed, 4 Dec 96 06:44 PST Received: from critter.tfs.com (localhost [127.0.0.1]) by critter.tfs.com (8.8.2/8.8.2) with ESMTP id JAA01929; Wed, 4 Dec 1996 09:31:33 +0100 (MET) To: "J.M. Chuang" cc: smp@csn.net (Steve Passe), smp@freebsd.org Subject: Re: Great ! In-reply-to: Your message of "Tue, 03 Dec 1996 16:52:28 -0400." <199612032052.QAA14333@bluenose.na.tuns.ca> Date: Wed, 04 Dec 1996 09:31:33 +0100 Message-ID: <1927.849688293@critter.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >I still have IDE problem :-(. > >For the latest smp kernel (supped this morning), if the second cpu >is not launched, everything works just fine but as soon as the >second cpu is activated, coredump shows up! Could I just launch an idea here ? In wd.c we use PIO to pull the data out of the IDE drives. Maybe we need to IPI while that goes on ? -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Wed Dec 4 08:16:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA24628 for smp-outgoing; Wed, 4 Dec 1996 08:16:24 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id IAA24618 for ; Wed, 4 Dec 1996 08:16:20 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVK3N-00025m-00; Wed, 4 Dec 1996 08:20:09 -0800 To: Terje Normann Marthinussen cc: smp@csn.net, smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 16:27:12 +0100." <199612041527.QAA04593@slibo.cc.uit.no> Date: Wed, 04 Dec 1996 08:20:09 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Terje Normann Marthinussen writes: > Just for the record, I did a continue in DDB just after this. > Now I wonder, what happens when you get into DDB, do you get the > code that generated the trap, or do get the code that cpu0 was at when > you got the trap? > > Tried after a reboot: > cpunumber = 0 > instruction pointer = 0x8:0xf010d65b > stack pointer = 0x10:0xefbfff68 > frame pointer = 0x10:0xefbfff6c > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, IOPL = 0 > current process = 6 (cpuidle1) > interrupt mask = > kernel: type 29 trap, code=0 ... The "type 29 trap" is the IPI used for SMP invalidates, I believe. I noticed that yours responds different than mine... did you hardwire the low-level to 2 cpus somewhere? -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Wed Dec 4 08:24:07 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA25246 for smp-outgoing; Wed, 4 Dec 1996 08:24:07 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id IAA25239 for ; Wed, 4 Dec 1996 08:24:03 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id RAA06683; Wed, 4 Dec 1996 17:23:36 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id RAA08146; Wed, 4 Dec 1996 17:23:35 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id RAA05827; Wed, 4 Dec 1996 17:23:34 +0100 (MET) Message-Id: <199612041623.RAA05827@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Erich Boleyn cc: smp@csn.net, smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 08:20:09 MET." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 17:23:33 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >I noticed that yours responds different than mine... did you hardwire the >low-level to 2 cpus somewhere? Yes, the current kernel is unable to start more than 2 CPU's in my system. Ends up with a panic in startAP in mp_machdep.c and a message that "Application Processor #3 failed!" I can either remove the panic(), or tell it to just start 2 CPU's (which I've done in this case). Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Wed Dec 4 08:33:37 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA26247 for smp-outgoing; Wed, 4 Dec 1996 08:33:37 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id IAA26218 for ; Wed, 4 Dec 1996 08:33:33 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id AAA00404; Thu, 5 Dec 1996 00:32:58 +0800 (WST) Message-Id: <199612041632.AAA00404@spinner.DIALix.COM> To: Erich Boleyn cc: Terje Normann Marthinussen , smp@csn.net, smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 08:20:09 PST." Date: Thu, 05 Dec 1996 00:32:58 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Erich Boleyn wrote: > > Terje Normann Marthinussen writes: > > > cpunumber = 0 > > instruction pointer = 0x8:0xf010d65b > > stack pointer = 0x10:0xefbfff68 > > frame pointer = 0x10:0xefbfff6c > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, def32 1, gran 1 > > processor eflags = interrupt enabled, IOPL = 0 > > current process = 6 (cpuidle1) > > interrupt mask = > > kernel: type 29 trap, code=0 > > ... > > The "type 29 trap" is the IPI used for SMP invalidates, I believe. We use ICU_OFFSET + 27 in the idt, so if anything it should be at vector 59. ICU_OFFSET is 32, so irq0 corresponds to idt slot #32. Entries 0-31 are partially defined by Intel and those that are not defined are reserved. Oh, wait a minute.. The IDT table is initialised with pointers to the "rsvd" handler which generates a #29 trap.. Did somebody mention not running with APIC_IO? SMP_INVLTLB is totally dependent on APIC_IO. Cheers, -Peter From owner-freebsd-smp Wed Dec 4 08:38:02 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA27468 for smp-outgoing; Wed, 4 Dec 1996 08:38:02 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id IAA27457 for ; Wed, 4 Dec 1996 08:37:59 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVKOc-00028j-00; Wed, 4 Dec 1996 08:42:06 -0800 To: smp@freebsd.org Subject: More than 2 CPUs Date: Wed, 04 Dec 1996 08:42:06 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk It strikes me that part of this more than 2 CPUs problem is sort of bogus. If you configure the kernel to only support 2 CPUs, it should just ignore any more than that, rather than causing a kernel panic. I just made a patch for "i386/i386/mp_machdep.c" which makes it ignore any more than the configured number of CPUs. I configured my kernel for 2 CPUs, and this now does the right thing. Please commit this (or something like it) to the SMP tree. When we get more than 2 CPU support, or want to test it, it is easy enough to bump the NCPU number in the config file. -------------------(start cvs diff sys/i386/i386/mp_machdep.c)--------------- Index: sys/i386/i386/mp_machdep.c =================================================================== RCS file: /usr/cvssup/sys/i386/i386/mp_machdep.c,v retrieving revision 1.28 diff -r1.28 mp_machdep.c 778,782c778,780 < /* add another AP to list */ < x = (*cpu)++; < if ( x == NCPU ) { < printf( "too many CPUs, increase 'NCPU'\n" ); < panic( "\n" ); --- > /* add another AP to list, if less than max number of CPUs */ > if ( x < (NCPU-1) ) { > x = (*cpu)++; 786,787c784,787 < CPU_TO_ID( x ) = entry->apicID; < ID_TO_CPU( entry->apicID ) = x; --- > if (x < NCPU) { > CPU_TO_ID( x ) = entry->apicID; > ID_TO_CPU( entry->apicID ) = x; > } --------------------(end cvs diff sys/i386/i386/mp_machdep.c)---------------- -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Wed Dec 4 08:38:51 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA27558 for smp-outgoing; Wed, 4 Dec 1996 08:38:51 -0800 (PST) Received: (from peter@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA27549 for freebsd-smp; Wed, 4 Dec 1996 08:38:50 -0800 (PST) Date: Wed, 4 Dec 1996 08:38:50 -0800 (PST) From: Peter Wemm Message-Id: <199612041638.IAA27549@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 mp_machdep.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk peter 96/12/04 08:38:49 Modified: i386/i386 mp_machdep.c Log: Revive the code that enforces the limit of 2 CPUs. Only start up one additional AP if more than one is present. Revision Changes Path 1.30 +9 -2 sys/i386/i386/mp_machdep.c From owner-freebsd-smp Wed Dec 4 08:47:12 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA28784 for smp-outgoing; Wed, 4 Dec 1996 08:47:12 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id IAA28773 for ; Wed, 4 Dec 1996 08:47:07 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVKXT-0002A3-00; Wed, 4 Dec 1996 08:51:15 -0800 To: smp@freebsd.org Subject: Greater than 2 CPUs... problem with my diff Date: Wed, 04 Dec 1996 08:51:15 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I just realized the diff I just sent was conceptually wrong (though it would probably work for all existing machines, and worked fine for mine). The real diff I should have sent for mp_machdep.c is included here, so please use that instead! -----------------(start cvs diff sys/i386/i386/mp_machdep.c)----------------- Index: sys/i386/i386/mp_machdep.c =================================================================== RCS file: /usr/cvssup/sys/i386/i386/mp_machdep.c,v retrieving revision 1.28 diff -r1.28 mp_machdep.c 778c778,779 < /* add another AP to list */ --- > /* add another AP to list, if less than max number of CPUs */ > if ( x == (NCPU-1) ) return; 780,783d780 < if ( x == NCPU ) { < printf( "too many CPUs, increase 'NCPU'\n" ); < panic( "\n" ); < } ------------------(end cvs diff sys/i386/i386/mp_machdep.c)------------------ -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Wed Dec 4 08:49:34 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA28992 for smp-outgoing; Wed, 4 Dec 1996 08:49:34 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id IAA28986 for ; Wed, 4 Dec 1996 08:49:30 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id JAA06293; Wed, 4 Dec 1996 09:48:28 -0700 Message-Id: <199612041648.JAA06293@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Erich Boleyn cc: Terje Normann Marthinussen , smp@freebsd.org, Peter Wemm Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 08:20:09 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 09:48:28 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >2 CPUs can't work without SMP_AUTOSTART enabled. SMP_AUTOSTART causes the grief that kept everyone cussing last week. I have a greatly imporved version almost ready for commit. I think I have an idea for a "semi" autostart that will both work and allow >2 CPUs. Should be ready in several hours. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Wed Dec 4 08:50:33 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA29068 for smp-outgoing; Wed, 4 Dec 1996 08:50:33 -0800 (PST) Received: from bacall.lodgenet.com (bacall.lodgenet.com [205.138.147.242]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id IAA29059 for ; Wed, 4 Dec 1996 08:50:29 -0800 (PST) Received: (from mail@localhost) by bacall.lodgenet.com (8.6.12/8.6.12) id KAA04604 for ; Wed, 4 Dec 1996 10:50:03 -0600 Received: from garbo.lodgenet.com(204.124.123.250) by bacall via smap (V1.3) id sma004586; Wed Dec 4 10:49:41 1996 Received: from jake.lodgenet.com (jake.lodgenet.com [10.0.11.30]) by garbo.lodgenet.com (8.6.12/8.6.9) with ESMTP id KAA24650 for ; Wed, 4 Dec 1996 10:49:48 -0600 Received: from jake.lodgenet.com (localhost [127.0.0.1]) by jake.lodgenet.com (8.8.3/8.6.12) with ESMTP id KAA18601 for ; Wed, 4 Dec 1996 10:49:59 -0600 (CST) Message-Id: <199612041649.KAA18601@jake.lodgenet.com> X-Mailer: exmh version 1.6.9 8/22/96 To: smp@freebsd.org Subject: INVLTLB and stuff Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 10:49:59 -0600 From: "Eric L. Hernes" Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi guys, I decided to go for it all and test with APIC_IO and SMP_INVLTLB first, then back off if necessary. during a kernel compile: (ttyp0@jake)$ vmstat -i interrupt total rate clk0 irq2 744193 431 rtc0 irq8 214594 124 pci irq11 17629 10 sc0 irq1 804 0 sio1 irq3 32319 18 lpt0 irq7 1 0 ed0 irq5 14927 8 sb0 irq9 321 0 stray irq13 1 0 Total 1024789 594 and the tail of dmesg: DEVFS: ready to run Enabled INTs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 13, imen: 0x00ffd401 SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... sio1: 1 more silo overflow (total 1) sio1: 1 more silo overflow (total 2) sio1: 1 more silo overflow (total 3) sio1: 1 more silo overflow (total 4) sio1: 25 more tty-level buffer overflows (total 25) sio1: 92 more tty-level buffer overflows (total 117) sio1: 25 more tty-level buffer overflows (total 142) sio1: 12 more tty-level buffer overflows (total 154) sio1: 11 more tty-level buffer overflows (total 165) sio1: 10 more tty-level buffer overflows (total 175) sio1: 1 more silo overflow (total 5) sio1: 1 more silo overflow (total 6) ed0: device timeout ed0: device timeout ed0: device timeout ed0: device timeout sio1 is the mouse. system responsiveness is very choppy while anything significant is running, even a `make depend' So I removed the SMP_INVLTLB. Is it legal to just comment out #define SMP_INVLTLB 1 in opt_smp_invltlb.h? Or do I need to re-config? kernel without SMP_INVLTLB: (ttyp1@jake)$ vmstat -i interrupt total rate clk0 irq2 1244898 1144 ^^^^ Holy Christmas, what's up with that? rtc0 irq8 139347 128 pci irq11 25834 23 fdc0 irq6 2962 2 sc0 irq1 3278 3 sio1 irq3 53092 48 ed0 irq5 34118 31 sb0 irq9 372 0 stray irq13 1 0 Total 1503902 1382 but no lost interrupts, and the network timeouts are gone. kernel compile with j1: 1265.95 real 728.45 user 280.91 sys kernel compile with j8: 658.73 real 866.27 user 256.26 sys the machine was doing other compiles and work at the same time, so this isn't a very great indication of performance, but it's looking better! eric. -- erich@lodgenet.com http://rrnet.com/~erich erich@rrnet.com From owner-freebsd-smp Wed Dec 4 08:56:31 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id IAA29734 for smp-outgoing; Wed, 4 Dec 1996 08:56:31 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id IAA29723 for ; Wed, 4 Dec 1996 08:56:26 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id AAA00583; Thu, 5 Dec 1996 00:56:05 +0800 (WST) Message-Id: <199612041656.AAA00583@spinner.DIALix.COM> To: Erich Boleyn cc: smp@freebsd.org Subject: Re: Greater than 2 CPUs... problem with my diff In-reply-to: Your message of "Wed, 04 Dec 1996 08:51:15 PST." Date: Thu, 05 Dec 1996 00:56:05 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Erich Boleyn wrote: > > I just realized the diff I just sent was conceptually wrong (though it > would probably work for all existing machines, and worked fine for mine). > The real diff I should have sent for mp_machdep.c is included here, so > please use that instead! > > -----------------(start cvs diff sys/i386/i386/mp_machdep.c)----------------- > Index: sys/i386/i386/mp_machdep.c > =================================================================== > RCS file: /usr/cvssup/sys/i386/i386/mp_machdep.c,v > retrieving revision 1.28 > diff -r1.28 mp_machdep.c > 778c778,779 > < /* add another AP to list */ > --- > > /* add another AP to list, if less than max number of CPUs */ > > if ( x == (NCPU-1) ) return; > 780,783d780 > < if ( x == NCPU ) { > < printf( "too many CPUs, increase 'NCPU'\n" ); > < panic( "\n" ); > < } > ------------------(end cvs diff sys/i386/i386/mp_machdep.c)------------------ Umm, where is 'x' initialised? it's very hard to see exactly what you're trying to do, a context diff would be a lot better... Cheers, -Peter From owner-freebsd-smp Wed Dec 4 09:05:31 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA01475 for smp-outgoing; Wed, 4 Dec 1996 09:05:31 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id JAA01470 for ; Wed, 4 Dec 1996 09:05:28 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVKp4-0002DV-00; Wed, 4 Dec 1996 09:09:26 -0800 To: Peter Wemm cc: smp@freebsd.org Subject: Re: Greater than 2 CPUs... problem with my diff In-reply-to: Your message of "Thu, 05 Dec 1996 00:56:05 +0800." <199612041656.AAA00583@spinner.DIALix.COM> Date: Wed, 04 Dec 1996 09:09:26 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Peter Wemm writes: > Erich Boleyn wrote: ... > Umm, where is 'x' initialised? it's very hard to see exactly what you're > trying to do, a context diff would be a lot better... Wow, you're right. I can't believe I completely spaced that. I noticed it again after I sent it. I'm also surprised it worked correctly... weird. Anyway, here's a context diff where x *is* initialized: ------------------------(start diff mp_machdep.c)------------------------- --- mp_machdep.c Tue Dec 3 19:06:09 1996 +++ /usr/src/sys/i386/i386/mp_machdep.c Wed Dec 4 09:05:34 1996 @@ -764,7 +764,7 @@ static void processorEntry( ProcEntry entry, int* cpu ) { - int x; + int x = *cpu; /* check for usability */ if ( !(entry->cpuFlags & PROCENTRY_FLAG_EN) ) return; @@ -775,12 +775,10 @@ x = 0; bootCpuId = entry->apicID; } else { - /* add another AP to list */ - x = (*cpu)++; - if ( x == NCPU ) { - printf( "too many CPUs, increase 'NCPU'\n" ); - panic( "\n" ); - } + /* add another AP to list, if less than max number of CPUs */ + x++; + if ( x == NCPU ) return; + *cpu = x; } CPU_TO_ID( x ) = entry->apicID; ------------------------(end diff mp_machdep.c)------------------------- -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Wed Dec 4 09:07:30 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA01620 for smp-outgoing; Wed, 4 Dec 1996 09:07:30 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id JAA01613 for ; Wed, 4 Dec 1996 09:07:25 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id SAA07529; Wed, 4 Dec 1996 18:06:54 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id SAA08306; Wed, 4 Dec 1996 18:06:53 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id SAA06907; Wed, 4 Dec 1996 18:06:52 +0100 (MET) Message-Id: <199612041706.SAA06907@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Peter Wemm cc: smp@csn.net, smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Thu, 05 Dec 1996 00:32:58 MET." <199612041632.AAA00404@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 18:06:52 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > >We use ICU_OFFSET + 27 in the idt, so if anything it should be at vector >59. ICU_OFFSET is 32, so irq0 corresponds to idt slot #32. Entries 0-31 >are partially defined by Intel and those that are not defined are reserved. > >Oh, wait a minute.. The IDT table is initialised with pointers to the >"rsvd" handler which generates a #29 trap.. Did somebody mention not >running with APIC_IO? SMP_INVLTLB is totally dependent on APIC_IO. Argh! No APIC_IO, that's right... Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Wed Dec 4 09:08:43 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA01747 for smp-outgoing; Wed, 4 Dec 1996 09:08:43 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id JAA01738 for ; Wed, 4 Dec 1996 09:08:40 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA06446 for ; Wed, 4 Dec 1996 10:08:37 -0700 Message-Id: <199612041708.KAA06446@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@freebsd.org Subject: Re: modified tools outside of kernel source tree In-reply-to: Your message of "Wed, 04 Dec 1996 22:46:17 +0800." <199612041446.WAA00462@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 10:08:37 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > I have a tweaked version of config that knows how to reserve device_id > slots for the ipi interrupts. > ... > I imagine Steve would also like to put his mptable.c code under the tools > directory too. > > Does this sound OK? yes, it'll get therec later today. we also need to add a doc/ directory so filing some of the descriptions of how all this stuff works. This would be a good job for a volunteer, taking those longer mailings where we dscribe in detail some aspect of the design/code and editing them into short 'topic' papers. Good start for building formal docs. Something I just don't have time for.. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Dec 4 09:11:35 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA01985 for smp-outgoing; Wed, 4 Dec 1996 09:11:35 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id JAA01973 for ; Wed, 4 Dec 1996 09:11:32 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA06466; Wed, 4 Dec 1996 10:10:40 -0700 Message-Id: <199612041710.KAA06466@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Erich Boleyn cc: Terje.N.Marthinussen@cc.uit.no, smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 06:59:47 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 10:10:39 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, entry #9 in the mptable database for reference: http://www.freebsd.org/~fsmp/SMP/mptable/mptable.html -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Dec 4 09:33:40 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA06262 for smp-outgoing; Wed, 4 Dec 1996 09:33:40 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id JAA06257 for ; Wed, 4 Dec 1996 09:33:37 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA06644; Wed, 4 Dec 1996 10:33:24 -0700 Message-Id: <199612041733.KAA06644@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terje Normann Marthinussen cc: smp@freebsd.org, Peter Wemm Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 13:35:40 +0100." <199612041235.NAA00889@slibo.cc.uit.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 10:33:24 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > Our 4 CPU Netserver works without SMP_INVLTLB and APIC_IO (the latter > has never worked). With SMP_INVLTLB, I get something similar the catch-22 time, I forgot all about your missing PCI INTs when using APIC_IO, without APIC_IO you don't get SMP_INVLTBL. I think the problem is that damn re-direction thingy for PCI INTs to ISA bus. If I don't get a patch to you by wend of day remind me. your system says the PCI disk controler uses: -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# INT conforms level 19 7 14 7 ^^^^^^^^ ^^^^^ Which is different from the ISA standard of active-hi/edge. So far our experiments suggested that for cards (PCI & EISA) redirected to ISA INTs we should ignore the entries and stick with ISA levels. This works on at least 1 EISA machine and 1 PCI machine that I have encountered. However the printout you sent earlier shows that your PCI card is completely loosing its INTs ubder APIC_IO, symptomatic of this pin being incorrectly programmed. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Dec 4 09:42:10 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id JAA08136 for smp-outgoing; Wed, 4 Dec 1996 09:42:10 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id JAA08122 for ; Wed, 4 Dec 1996 09:42:06 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVLOW-0002Ib-00; Wed, 4 Dec 1996 09:46:04 -0800 To: peter@spinner.dialix.com cc: smp@freebsd.org Subject: Get running with 2 CPUs only! (finally...) Date: Wed, 04 Dec 1996 09:46:04 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk It turns out that in the patch I sent, which was making sure "x" was initialized in the "processorEntry" routine, had an off-by-one error, as the CPU numbering was 1-based, and not 0-based, as I thought. Sigh. Anyway, I tried this kernel, and it returns the right thing from dmesg (only 2 CPUs probed and activated), the 2nd CPU activates and is now running on my box, etc. I even performed the diff on the kernel running 2-CPU. With this and NCPU = 2, I'm running with APIC_IO + SMP_INVLTLB, and 2 CPUs are up and running (I've been doing cvs stuff, compiles, etc. with no problems for the last 15 minutes). Thanks for all the great work, guys! Particularly now that TLB invalidates propagate, I feel much more comfortable running things. (and of course, it works on my test P6 box ;-). So, here is the correct and tested patch for limiting to NCPU cpus with no kernel panics (if you didn't see it and fix it already). This is against the cvs tree from this morning before I made any changes: ---------------------------(start diff mp_machdep.c)---------------------- --- mp_machdep.c Tue Dec 3 19:06:09 1996 +++ /usr/src/sys/i386/i386/mp_machdep.c Wed Dec 4 09:35:14 1996 @@ -764,7 +764,7 @@ static void processorEntry( ProcEntry entry, int* cpu ) { - int x; + int x = *cpu; /* check for usability */ if ( !(entry->cpuFlags & PROCENTRY_FLAG_EN) ) return; @@ -775,12 +775,9 @@ x = 0; bootCpuId = entry->apicID; } else { - /* add another AP to list */ - x = (*cpu)++; - if ( x == NCPU ) { - printf( "too many CPUs, increase 'NCPU'\n" ); - panic( "\n" ); - } + /* add another AP to list, if less than max number of CPUs */ + if ( x == NCPU ) return; + *cpu = x + 1; } CPU_TO_ID( x ) = entry->apicID; ---------------------------(end diff mp_machdep.c)---------------------- -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Wed Dec 4 10:15:52 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA13844 for smp-outgoing; Wed, 4 Dec 1996 10:15:52 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id KAA13839 for ; Wed, 4 Dec 1996 10:15:49 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id MAA05291 for ; Wed, 4 Dec 1996 12:15:39 -0600 (CST) Message-Id: <199612041815.MAA05291@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: freebsd-smp@freebsd.org Subject: make -j? questions.. Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 12:15:39 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk First of all, I remember seeing here recently that there were some problems, and I was wondering if they were fixed. Also, I know sometimes that it would fail almost at random, if you have -j set higher. It almost seemed as if there was some sort of dependancy issue that was ignored by the parallelism. Is this still the case? It would be nice if the -j option was suitable for make worlds and such. :) --Chris Csanady From owner-freebsd-smp Wed Dec 4 10:38:29 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id KAA15064 for smp-outgoing; Wed, 4 Dec 1996 10:38:29 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id KAA15059 for ; Wed, 4 Dec 1996 10:38:26 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id LAA07089; Wed, 4 Dec 1996 11:38:15 -0700 Message-Id: <199612041838.LAA07089@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Chris Csanady cc: freebsd-smp@freebsd.org Subject: Re: make -j? questions.. In-reply-to: Your message of "Wed, 04 Dec 1996 12:15:39 CST." <199612041815.MAA05291@friley216.res.iastate.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 11:38:15 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > First of all, I remember seeing here recently that there were some problems, > and I was wondering if they were fixed. Also, I know sometimes that it would > fail almost at random, if you have -j set higher. It almost seemed as if there > was some sort of dependancy issue that was ignored by the parallelism. Is > this still the case? that was me. the problem is make clean removes "?". (can't remember what "?" is, to find it do "make clean;make -j4", my machine is wedged at the moment...) make depend builds "?" make builds "?", but a different verson than "make depend"!!! something about what files are around when "?" is built. I think the solution is to make "make" realize that it needs to "make depend" before anything else if ".depend" doesn't exist --- > It would be nice if the -j option was suitable for make worlds and such. :) so as long as you do a "make depend" between any "make clean" and "make" you are OK -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Dec 4 11:02:06 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id LAA15930 for smp-outgoing; Wed, 4 Dec 1996 11:02:06 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id LAA15925 for ; Wed, 4 Dec 1996 11:02:00 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id DAA00705 for ; Thu, 5 Dec 1996 03:01:55 +0800 (WST) Message-Id: <199612041901.DAA00705@spinner.DIALix.COM> To: smp@freebsd.org Subject: prototype diffs for config(8) Date: Thu, 05 Dec 1996 03:01:54 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk If anbody wants to try these out, they should add the counters for the 4 ipi interrupts to 'vmstat -i' and 'systat -vmstat'. -Peter Index: mkglue.c =================================================================== RCS file: /home/ncvs/src/usr.sbin/config/mkglue.c,v retrieving revision 1.10 diff -c -r1.10 mkglue.c *** mkglue.c 1996/06/02 17:21:57 1.10 --- mkglue.c 1996/12/04 13:55:35 *************** *** 363,369 **** fprintf(fp, "pci irqnn\\0\\\n"); fprintf(fp, "pci irqnn\\0\\\n"); fprintf(fp, "pci irqnn\\0\\\n"); ! dev_id = 6; vector_devtab(fp, "bio", &dev_id); vector_devtab(fp, "tty", &dev_id); vector_devtab(fp, "net", &dev_id); --- 363,373 ---- fprintf(fp, "pci irqnn\\0\\\n"); fprintf(fp, "pci irqnn\\0\\\n"); fprintf(fp, "pci irqnn\\0\\\n"); ! fprintf(fp, "ipi irqnn\\0\\\n"); ! fprintf(fp, "ipi irqnn\\0\\\n"); ! fprintf(fp, "ipi irqnn\\0\\\n"); ! fprintf(fp, "ipi irqnn\\0\\\n"); ! dev_id = 10; vector_devtab(fp, "bio", &dev_id); vector_devtab(fp, "tty", &dev_id); vector_devtab(fp, "net", &dev_id); Index: mkioconf.c =================================================================== RCS file: /home/ncvs/src/usr.sbin/config/mkioconf.c,v retrieving revision 1.25 diff -c -r1.25 mkioconf.c *** mkioconf.c 1996/06/02 17:21:58 1.25 --- mkioconf.c 1996/12/04 13:22:51 *************** *** 682,688 **** if (printed) fprintf(fp1, "\n"); } ! dev_id = 6; /* XXX must match mkglue.c */ isa_devtab(fp, "bio", &dev_id); if (seen_wdc) isa_biotab(fp, "wdc"); --- 682,688 ---- if (printed) fprintf(fp1, "\n"); } ! dev_id = 10; /* XXX must match mkglue.c */ isa_devtab(fp, "bio", &dev_id); if (seen_wdc) isa_biotab(fp, "wdc"); == end == From owner-freebsd-smp Wed Dec 4 12:03:20 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id MAA19949 for smp-outgoing; Wed, 4 Dec 1996 12:03:20 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id MAA19887; Wed, 4 Dec 1996 12:03:14 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id OAA00722; Wed, 4 Dec 1996 14:03:13 -0600 (CST) Message-Id: <199612042003.OAA00722@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: freebsd-smp@freebsd.org cc: current@freebsd.org Subject: Possible file system damage.. Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 14:03:13 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I've just build another SMP kernel with APIC_IO and SMP_INVLTLB, and so far, everything seems to be working just fine. :) The only thing that bothered me was this message I got upon booting the SMP kernel: Dec 4 12:17:46 friley216 /kernel: bad block 2240515, ino 20 Dec 4 12:17:46 friley216 /kernel: pid 112 (kvm_mkdb), uid 0 on /var: bad block I looked at the code, and this looks as if ffs tried to use a block outside the limits of the file system. This sounds to me more like a file system problem, but I thought I'd check here first. It doesnt sound like it hurt anything, but it does worry me a bit. Somehow, invalid data got into in inode on a file system that was cleanly mounted and unmounted. --Chris Csanady From owner-freebsd-smp Wed Dec 4 13:37:54 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id NAA15073 for smp-outgoing; Wed, 4 Dec 1996 13:37:54 -0800 (PST) Received: from soleil.uvsq.fr (soleil.uvsq.fr [193.51.24.1]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id NAA15065 for ; Wed, 4 Dec 1996 13:37:51 -0800 (PST) Received: from arts.ratp.fr (arts.ratp.fr [193.106.40.1]) by soleil.uvsq.fr (8.8.3/jtpda-5.2) with ESMTP id WAA07967 ; Wed, 4 Dec 1996 22:37:46 +0100 (MET) Received: by arts.ratp.fr id WAA10045 ; Wed, 4 Dec 1996 22:37:42 +0100 (MET) Received: from minos.noisy.ratp by arts.ratp.fr with SMTP id SAA010043 for ; Wed Dec 4 22:37:21 1996 Received: by minos.noisy.ratp id WAA22931 ; Wed, 4 Dec 1996 22:37:22 +0100 (MET) From: Janick.Taillandier@ratp.fr (Janick TAILLANDIER) Message-Id: <199612042137.WAA22931@minos.noisy.ratp> Subject: Trap 12 To: freebsd-smp@freebsd.org Date: Wed, 4 Dec 1996 22:37:21 +0100 (MET) X-Mailer: ELM [version 2.4 PL24 ME8] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hello, I have installed the latest patches and I am still having trap 12 a few seconds after starting the second CPU. Everything seems ok with only one CPU. The machine is a Dell Optiplex GXpro with 2 P6 200 MHz, 32MB, SCSI disk on Adaptec 2940UW. I am using the following options : options SMP options "NCPU=2" options APIC_IO options "NBUS=3" options "NAPIC=1" options "NINTR=36" options SMP_INVLTLB With all these options, here is what I get : Stopped at _pmap_enter+0x8f: movl 0(%ecx),%ecx trace gives: _pmap_enter(f13ad064,efbfd000,17c8000,7,0) at _pmap_enter+0x8f _vm_fault(f13ad000,efbfd000,3,0,0) at _vm_fault+xxd`b _trap_pfault(efbfffbc,1) at _trap_pfault+0xd4 _trap(27,27,50274,1,efbfd5F4) at _trap+0x14b calltrap() at calltrap+0x1a --- trap 12, eip = 0xe9fc, ebp =0xefbfd5f4 --- --- curproc = 0xf13e9a00, pid = 1561 --- I have similar crash without APIC_IO or SMP_INVLTLB. Janick From owner-freebsd-smp Wed Dec 4 14:33:04 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id OAA19805 for smp-outgoing; Wed, 4 Dec 1996 14:33:04 -0800 (PST) Received: (from fsmp@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id OAA19798 for freebsd-smp; Wed, 4 Dec 1996 14:33:02 -0800 (PST) Date: Wed, 4 Dec 1996 14:33:02 -0800 (PST) From: Steve Passe Message-Id: <199612042233.OAA19798@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/kern init_smp.c sys/i386/i386 mp_machdep.c mpapic.c mpboot.s mplock.s swtch.s sys/i386/include smp.h Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk fsmp 96/12/04 14:33:02 Modified: kern init_smp.c Log: cleanup of SMP_AUTOSTART code. most of it is now default, hopefully supporting >2 CPUs. SMP_AUTOSTART is still an option, and still is BROKEN. cleanup of APIC initialization code. Revision Changes Path 1.43 +54 -43 sys/kern/init_smp.c Modified: i386/include smp.h Log: function declaration for cleanup of APIC initialization code. Revision Changes Path 1.24 +2 -2 sys/i386/include/smp.h Modified: i386/i386 mp_machdep.c mpapic.c mpboot.s mplock.s swtch.s Log: cleanup of APIC initialization code in mpboot.s. cleanup of mp_lock handling in swtch.s. new functionality for initializing AP APICs. better support for SMP_AUTOSTART, but SMP_AUTOSTART still BROKEN. 3/4 CPUs hopefully now start correctly with "sysctl -w kern.smp_active=4" Note: I have NOT yet merged the various patches sent today for throttling to 2 CPUs when "options SMP=2" is set. Revision Changes Path 1.31 +2 -2 sys/i386/i386/mp_machdep.c 1.23 +87 -80 sys/i386/i386/mpapic.c 1.14 +20 -45 sys/i386/i386/mpboot.s 1.15 +13 -5 sys/i386/i386/mplock.s 1.30 +19 -8 sys/i386/i386/swtch.s From owner-freebsd-smp Wed Dec 4 15:02:50 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id PAA20996 for smp-outgoing; Wed, 4 Dec 1996 15:02:50 -0800 (PST) Received: from gargoyle.bazzle.com (gargoyle.bazzle.com [206.103.246.190]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA20989 for ; Wed, 4 Dec 1996 15:02:44 -0800 (PST) Received: from gargoyle.bazzle.com (localhost [127.0.0.1]) by gargoyle.bazzle.com (8.8.3/8.6.12) with SMTP id SAA05334; Wed, 4 Dec 1996 18:02:35 -0500 (EST) Date: Wed, 4 Dec 1996 18:02:35 -0500 (EST) From: "Eric J. Chet" To: Chris Csanady cc: freebsd-smp@freebsd.org Subject: Re: make -j? questions.. In-Reply-To: <199612041815.MAA05291@friley216.res.iastate.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Wed, 4 Dec 1996, Chris Csanady wrote: > First of all, I remember seeing here recently that there were some problems, > and I was wondering if they were fixed. Also, I know sometimes that it would > fail almost at random, if you have -j set higher. It almost seemed as if there > was some sort of dependancy issue that was ignored by the parallelism. Is > this still the case? > > It would be nice if the -j option was suitable for make worlds and such. :) > > --Chris Csanady > > > Hello When I make a kernel with "make -j8" and I have not done a "make depend" first the parallel make usually fails. I have not attemped a parallel make world. Eric J. Chet - ejc@bazzle.com From owner-freebsd-smp Wed Dec 4 15:03:33 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id PAA21014 for smp-outgoing; Wed, 4 Dec 1996 15:03:33 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA21009 for ; Wed, 4 Dec 1996 15:03:30 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id TAA18676 for smp@freebsd.org; Wed, 4 Dec 1996 19:01:05 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612042301.TAA18676@bluenose.na.tuns.ca> Subject: Good News & Bad News To: smp@freebsd.org Date: Wed, 4 Dec 1996 19:01:05 -0400 (AST) X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, With the current smp kernel+ SMP_INVLTLB, the good news is that Tomcat III+IDE is able to `make world' successfully with FreeBSD-current source tree. With the same kernel, the Titan Pro is still suffering with the core dump of using IDE right after the second CPU is launched. If the SCSI is used to boot the system, there is no coredump but the system reboots itself unpredictably shortly after the second CPU is activated. Jim From owner-freebsd-smp Wed Dec 4 15:18:49 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id PAA21575 for smp-outgoing; Wed, 4 Dec 1996 15:18:49 -0800 (PST) Received: from cs.utah.edu (cs.utah.edu [128.110.4.21]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA21570 for ; Wed, 4 Dec 1996 15:18:43 -0800 (PST) Received: from fast.cs.utah.edu by cs.utah.edu (8.8.3/utah-2.21-cs) id QAA00452; Wed, 4 Dec 1996 16:18:35 -0700 (MST) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id QAA23894; Wed, 4 Dec 1996 16:18:34 -0700 Date: Wed, 4 Dec 1996 16:18:34 -0700 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199612042318.QAA23894@fast.cs.utah.edu> To: smp@bluenose.na.tuns.ca, smp@freebsd.org Subject: Re: Good News & Bad News Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >With the current smp kernel+ SMP_INVLTLB, the good news is that Tomcat III+IDE >is able to `make world' successfully with FreeBSD-current source tree. > >With the same kernel, the Titan Pro is still suffering with >the core dump of using IDE right after the second CPU is launched. >If the SCSI is used to boot the system, there is no coredump but >the system reboots itself unpredictably shortly after the second >CPU is activated. Just a thought, but how many of the machines besides the Tyan Titan Pro have the BSP at other than CPU #0? The Titan Pro's BSD is (physically) Processor #1. This is just the most obvious difference between the Titan Pro and the other boards. Kevin From owner-freebsd-smp Wed Dec 4 15:41:28 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id PAA22510 for smp-outgoing; Wed, 4 Dec 1996 15:41:28 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id PAA22502 for ; Wed, 4 Dec 1996 15:41:21 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id QAA08544; Wed, 4 Dec 1996 16:40:45 -0700 Message-Id: <199612042340.QAA08544@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: smp@bluenose.na.tuns.ca, smp@freebsd.org Subject: Re: Good News & Bad News In-reply-to: Your message of "Wed, 04 Dec 1996 16:18:34 MST." <199612042318.QAA23894@fast.cs.utah.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 04 Dec 1996 16:40:45 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > >With the current smp kernel+ SMP_INVLTLB, the good news is that Tomcat III+IDE > >is able to `make world' successfully with FreeBSD-current source tree. > > > >With the same kernel, the Titan Pro is still suffering with > >the core dump of using IDE right after the second CPU is launched. > >If the SCSI is used to boot the system, there is no coredump but > >the system reboots itself unpredictably shortly after the second > >CPU is activated. > > Just a thought, but how many of the machines besides the Tyan Titan > Pro have the BSP at other than CPU #0? The Titan Pro's BSD is > (physically) Processor #1. This is just the most obvious difference > between the Titan Pro and the other boards. from the mptab;e database: Asus P/I-P65UP5 Supermicro P6DNE does anyone running these machines have experience with the recent SMP kernel? -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Dec 4 15:54:54 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id PAA23080 for smp-outgoing; Wed, 4 Dec 1996 15:54:54 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA23065 for ; Wed, 4 Dec 1996 15:54:52 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id TAA19000; Wed, 4 Dec 1996 19:52:22 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612042352.TAA19000@bluenose.na.tuns.ca> Subject: Re: Good News & Bad News To: smp@csn.net (Steve Passe) Date: Wed, 4 Dec 1996 19:52:21 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612042329.QAA08473@clem.systemsix.com> from Steve Passe at "Dec 4, 96 04:29:50 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > > I would try setting memory speed more conservatively in your BIOS just > as a test. Ie bump memory speed from 60ns to 70ns and see if that makes > it more stable. Also, if there is write-back vs write-thru cache settings > try changing them. After setting memory speed to 70ns, no luck. (there is no write-back vs write-thru cache settings for Titan pro) Jim From owner-freebsd-smp Wed Dec 4 16:07:24 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id QAA23627 for smp-outgoing; Wed, 4 Dec 1996 16:07:24 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id QAA23620 for ; Wed, 4 Dec 1996 16:07:20 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id BAA14133; Thu, 5 Dec 1996 01:07:15 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id BAA10669; Thu, 5 Dec 1996 01:07:14 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id BAA16429; Thu, 5 Dec 1996 01:07:13 +0100 (MET) Message-Id: <199612050007.BAA16429@slibo.cc.uit.no> X-Mailer: exmh version 1.6.9 8/22/96 To: Steve Passe cc: smp@freebsd.org Subject: Re: cvs commit: sys/kern init_smp.c sys/i386/i386 mp_machdep.c mpapic.c mpboot.s mplock.s swtch.s sys/i386/include smp.h In-reply-to: Your message of "Wed, 04 Dec 1996 14:33:02 MET." <199612042233.OAA19798@freefall.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 01:07:12 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Starting >2CPUs at my computer still fails. (remove the >2 check as well, still no luck). Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Wed Dec 4 21:03:53 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id VAA14129 for smp-outgoing; Wed, 4 Dec 1996 21:03:53 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id VAA14119 for ; Wed, 4 Dec 1996 21:03:45 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id WAA10189; Wed, 4 Dec 1996 22:03:29 -0700 Message-Id: <199612050503.WAA10189@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@freebsd.org cc: Terje Normann Marthinussen , Erich Boleyn Subject: Re: Get running with 2 CPUs only! (finally...) In-reply-to: Your message of "Wed, 04 Dec 1996 09:46:04 PST." Mime-Version: 1.0 Content-Type: text/plain Date: Wed, 04 Dec 1996 22:03:29 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, The following 2 patches for i386/i386/mp_machdep.c and i386/i386/mpboot.s attempt to track how far we get when launching CPUs 3 & 4. What they do is set initial values of 99 in 6 consecutive bytes of CMOS ram. The advantage of this is that it is all done via outb instructions, so they can be done very early in the AP boot process, before they have valid stack, etc. This should tell us how far, if at all the APs run. The results are printed very near the top of the output, before the dmesg buffer works, so a serial debug would be helpful. you could also put a cngetc() call in after the CHECK_PRINT()s to stop them. The expected output of each AP is: ... check: 99, 99, 99, 99, 99, 99 check: 1, 2, 3, 4, 5, 99 Sizing memory.. init386 done CR0 = 80000011 ---------------------------------- cut --------------------------------------- *** mpboot.s 1996/12/04 22:32:56 1.14 --- mpboot.s 1996/12/05 04:23:14 *************** *** 31,37 **** * mpboot.s: FreeBSD machine support for the Intel MP Spec * multiprocessor systems. * ! * $Id: mpboot.s,v 1.14 1996/12/04 22:32:56 fsmp Exp $ */ #include "opt_smp_autostart.h" --- 31,37 ---- * mpboot.s: FreeBSD machine support for the Intel MP Spec * multiprocessor systems. * ! * $Id: mpboot.s,v 1.14 1996/12/04 22:32:56 fsmp Exp fsmp $ */ #include "opt_smp_autostart.h" *************** *** 42,53 **** --- 42,62 ---- #include "assym.s" + #if 1 + #define CHECK(A,D) \ + movb $(A),%al ; \ + outb %al,$0x70 ; \ + movb $(D),%al ; \ + outb %al,$0x71 + #endif + /* * the APs enter here from their trampoline code (bootMP, below) */ .align 4 NON_GPROF_ENTRY(MPentry) + CHECK(0x36,3) movl $mp_stk-KERNBASE,%esp /* mp boot stack end loc. */ /* Now enable paging mode */ movl _IdlePTD-KERNBASE, %eax *************** *** 65,71 **** --- 74,82 ---- * Wait for the booting CPU to signal startup */ mp_begin: /* now running relocated at KERNBASE */ + CHECK(0x37,4) call _init_secondary /* load i386 tables */ + CHECK(0x38,5) /* disable the APIC, just to be SURE */ movl _apic_base, %esi *************** *** 87,92 **** --- 98,104 ---- xchgb %al, bootlock /* xchg is implicitly locked */ cmpb $BL_SET, %al /* was is set? */ jz 1b /* yes, keep trying... */ + CHECK(0x39,6) /* Now, let's do some REAL WORK :-) */ call _secondary_main *************** *** 121,126 **** --- 133,139 ---- NON_GPROF_ENTRY(bootMP) cli + CHECK(0x34,1) /* First guarantee a 'clean slate' */ data32 xorl %eax, %eax *************** *** 176,181 **** --- 189,195 ---- lret protmode: + CHECK(0x35,2) /* * we are NOW running for the first time with %eip * having the full physical address, BUT we still ---------------------------------- cut --------------------------------------- ---------------------------------- cut --------------------------------------- *** mp_machdep.c 1996/12/04 22:32:53 1.31 --- mp_machdep.c 1996/12/05 04:44:06 *************** *** 22,28 **** * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * ! * $Id: mp_machdep.c,v 1.31 1996/12/04 22:32:53 fsmp Exp $ */ #include "opt_smp_invltlb.h" --- 22,28 ---- * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * ! * $Id: mp_machdep.c,v 1.31 1996/12/04 22:32:53 fsmp Exp fsmp $ */ #include "opt_smp_invltlb.h" *************** *** 1463,1468 **** --- 1463,1471 ---- } + #define CHECK_READ(A) (outb( CMOS_REG, (A) ), inb( CMOS_DATA )) + #define CHECK_WRITE(A,D) (outb( CMOS_REG, (A) ), outb( CMOS_DATA, (D) )) + /* * this function starts the AP (application processor) identified * by the APIC ID 'physicalCpu'. It does quite a "song and dance" *************** *** 1470,1481 **** --- 1473,1498 ---- * of the different hardware we might encounter. It ain't pretty, * but it seems to work. */ + #define CHECK_PRINT() \ + printf( "check: %d, %d, %d, %d, %d, %d\n", \ + CHECK_READ( 0x34 ), CHECK_READ( 0x35 ), \ + CHECK_READ( 0x36 ), CHECK_READ( 0x37 ), \ + CHECK_READ( 0x38 ), CHECK_READ( 0x39 ) ) static int startAP( int physicalCpu, int vector ) { int cpus; u_long icrLo, icrHi; + #if 1 + CHECK_WRITE( 0x34, 99 ); + CHECK_WRITE( 0x35, 99 ); + CHECK_WRITE( 0x36, 99 ); + CHECK_WRITE( 0x37, 99 ); + CHECK_WRITE( 0x38, 99 ); + CHECK_WRITE( 0x39, 99 ); + CHECK_PRINT(); + #endif /* used as a watchpoint to signal AP startup */ cpus = mp_ncpus; *************** *** 1541,1551 **** /* wait for it to start */ setApicTimer( 5000000 ); while ( readApicTimer() ) ! if ( mp_ncpus > cpus ) return 1; /* it started! */ ! #if 1 /* better panic as the AP may be running loose somewhere */ printf( "Application Processor #%d failed!\n", physicalCpu ); panic( "\n" ); #endif /* 1 */ --- 1558,1575 ---- /* wait for it to start */ setApicTimer( 5000000 ); while ( readApicTimer() ) ! if ( mp_ncpus > cpus ) { #if 1 + CHECK_PRINT(); + #endif + return 1; /* it started! */ + } + /* better panic as the AP may be running loose somewhere */ printf( "Application Processor #%d failed!\n", physicalCpu ); + #if 1 + CHECK_PRINT(); + #else panic( "\n" ); #endif /* 1 */ ---------------------------------- cut --------------------------------------- -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Dec 4 22:27:00 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id WAA17735 for smp-outgoing; Wed, 4 Dec 1996 22:27:00 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id WAA17730 for ; Wed, 4 Dec 1996 22:26:58 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id AAA03220 for ; Thu, 5 Dec 1996 00:26:57 -0600 (CST) Message-Id: <199612050626.AAA03220@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: freebsd-smp@freebsd.org Subject: make locking more generic? Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 00:26:56 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Could we possibly make our global semaphore a bit more generic? I guess this is what I had in mind.. It would allow us to create more locks. We could then easily give the run queues their own lock. This would be a fairly large win for a cpu bound workload I believe. Instead of having kernel entry block, when the kernel is running on another cpu, we could immediately switch to another process and do something useful. This would all be quite simple to implement, imho. We would then also be able to move the global kernel entry lock out into the syscalls, and further fan out the lock structure from there. This is something that I'd like to work on soon. :) The reason I was wanting the spin locks earlier was that to move the locks into the syscalls, we would need to make the interrupt, trap, and exception code re-entrant. I think this would be fairly trivial, and properly done with a few spin locks. Although, using semaphores would work fine as for now--and actually I think some of it was in place last time i checked. If you like, I could submit some patches for the global lock changes. I think I understand enough assembly now to get that right, although anyone else is more than welcome to do it.. :) --Chris Csanady From owner-freebsd-smp Wed Dec 4 22:50:29 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id WAA18578 for smp-outgoing; Wed, 4 Dec 1996 22:50:29 -0800 (PST) Received: from dyson.iquest.net (dyson.iquest.net [198.70.144.127]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id WAA18570 for ; Wed, 4 Dec 1996 22:50:26 -0800 (PST) Received: (from root@localhost) by dyson.iquest.net (8.8.2/8.6.9) id BAA04384; Thu, 5 Dec 1996 01:50:19 -0500 (EST) From: "John S. Dyson" Message-Id: <199612050650.BAA04384@dyson.iquest.net> Subject: Re: make locking more generic? To: ccsanady@friley216.res.iastate.edu (Chris Csanady) Date: Thu, 5 Dec 1996 01:50:18 -0500 (EST) Cc: freebsd-smp@FreeBSD.org In-Reply-To: <199612050626.AAA03220@friley216.res.iastate.edu> from "Chris Csanady" at Dec 5, 96 00:26:56 am Reply-To: dyson@FreeBSD.org X-Mailer: ELM [version 2.4 PL24 ME8] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > Could we possibly make our global semaphore a bit more generic? I guess this > is what I had in mind.. > FYI, I have been thinking about locking in the VM and VFS subsystems. I am sure that you will need to be taking input from various sources, in the VM system in particular, it will be likely that we might need to re-organize some of the code to fix deadlocks. Eventually, if we want good fine grained locking, we'll need to work out a clean hierarchy. I am also very hopeful of being able to build the single cpu code with very very little additional overhead, and SMP code with whatever added overhead necessary for locking code. If any of you SMP guys have any words of wisdom, or documentation sources for SMP technology, I would really like to hear from you with pointers. Also, I would sure like to hear about the direction that you are planning to go, so I can learn what I need to help. I am going to be getting to the VM documentation, and that will be a good time to in parallel look at feasible locking hierarchies. When benchmarking and evaluating the performance of the system, assuming the images/files are already in memory the VM routines are pretty high on the list as to cpu usage. John dyson@freebsd.org From owner-freebsd-smp Thu Dec 5 01:08:30 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id BAA23841 for smp-outgoing; Thu, 5 Dec 1996 01:08:30 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id BAA23808; Thu, 5 Dec 1996 01:08:25 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id CAA11365; Thu, 5 Dec 1996 02:08:21 -0700 Message-Id: <199612050908.CAA11365@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: dyson@FreeBSD.org cc: ccsanady@friley216.res.iastate.edu (Chris Csanady), freebsd-smp@FreeBSD.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 01:50:18 EST." <199612050650.BAA04384@dyson.iquest.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 02:08:20 -0700 Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk Hi, >If any of you SMP guys have any words of wisdom, or documentation sources >for SMP technology, I would really like to hear from you with pointers. Also, http://www.freebsd.org/~fsmp/SMP/books.html http://www.freebsd.org/~fsmp/SMP/threads.html -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 01:25:18 1996 Return-Path: owner-smp Received: (from root@localhost) by freefall.freebsd.org (8.8.3/8.7.3) id BAA26003 for smp-outgoing; Thu, 5 Dec 1996 01:25:18 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id BAA25995 for ; Thu, 5 Dec 1996 01:25:11 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id RAA07355 for ; Thu, 5 Dec 1996 17:25:02 +0800 (WST) Message-Id: <199612050925.RAA07355@spinner.DIALix.COM> To: smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 02:08:20 MST." <199612050908.CAA11365@clem.systemsix.com> Date: Thu, 05 Dec 1996 17:25:01 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve Passe wrote: > Hi, > > >If any of you SMP guys have any words of wisdom, or documentation sources > >for SMP technology, I would really like to hear from you with pointers. Als o, > > http://www.freebsd.org/~fsmp/SMP/books.html > http://www.freebsd.org/~fsmp/SMP/threads.html > -- > Steve Passe | powered by > smp@csn.net | FreeBSD There are also several "classes" of locks to consider. The version that we use to guard the entry/exit points of the kernel allows recursion and hence has a "nest count" which accounts for it's complexity and it's impact on the task switch code. By recursion, I mean that a process can enter the kernel lock, call something which ends up in copyin() and takes another fault, which re-enters the kernel lock again. it needs to keep track of how many times each process has entered the kernel so that it can unwind it correctly. Then there's a simple boolean lock. It's a simple request/free lock that does not care which cpu is in it at the time and does not allow recursion. ENTRY(boot_lock) /* This is the Intel reccomended semaphore method */ movb $0xff, %al 2: xchgb %al, bootlock /* xchg is implicitly locked */ cmpb $0xff, %al jz 2b ret ENTRY(boot_unlock) movb $0, %al xchgb %al, bootlock /* xchg is implicitly locked */ ret /* initial value is 0xff, or "busy" */ .globl bootlock bootlock: .byte 0xff Of course, there would be nothing stopping us using this simple lock primative to protect a counting semaphore that allowed recursion and didn't have to worry about atomic code. Then there's the 'lock ; bts ...' (bit test and set) and so on instructions. 4.4Lite2 has a lock 'manager' as well, that uses a simple lock to protect the complex locking structures. There is a good description of how this all fits together in a Lite2 context somewhere on the web. I think this is on Steve's pages, if not I'll have to check my bookmarks. There are other lock managers around, I believe Terry has mentioned one or two tree-based managers as well - these can be used to control the deadlock risks etc. Having a single lock is nice and convenient at this stage of the game, since we do not have to worry so much about kernel deadlocks while we figure out the fundamental core functionality. (eg: APIC_IO, IPI's, TLB sync, scheduling, starup/shutdown, idle loops, NCPU > 2 support etc) Cheers, -Peter From owner-freebsd-smp Thu Dec 5 06:47:10 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id GAA11078 for smp-outgoing; Thu, 5 Dec 1996 06:47:10 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id GAA11071 for ; Thu, 5 Dec 1996 06:47:06 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id IAA04684; Thu, 5 Dec 1996 08:47:04 -0600 (CST) Message-Id: <199612051447.IAA04684@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: Peter Wemm cc: smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of Thu, 05 Dec 1996 17:25:01 +0800. <199612050925.RAA07355@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 08:46:59 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >Steve Passe wrote: >> Hi, >> >> >If any of you SMP guys have any words of wisdom, or documentation sources >> >for SMP technology, I would really like to hear from you with pointers. Al s > o, >> >> http://www.freebsd.org/~fsmp/SMP/books.html >> http://www.freebsd.org/~fsmp/SMP/threads.html >> -- >> Steve Passe | powered by >> smp@csn.net | FreeBSD > >There are also several "classes" of locks to consider. The version >that we use to guard the entry/exit points of the kernel allows recursion >and hence has a "nest count" which accounts for it's complexity and it's >impact on the task switch code. By recursion, I mean that a process >can enter the kernel lock, call something which ends up in copyin() >and takes another fault, which re-enters the kernel lock again. it needs >to keep track of how many times each process has entered the kernel so >that it can unwind it correctly. > >Then there's a simple boolean lock. It's a simple request/free lock >that does not care which cpu is in it at the time and does not allow >recursion. > >ENTRY(boot_lock) > /* This is the Intel reccomended semaphore method */ > movb $0xff, %al >2: > xchgb %al, bootlock /* xchg is implicitly locked */ > cmpb $0xff, %al > jz 2b > ret > >ENTRY(boot_unlock) > movb $0, %al > xchgb %al, bootlock /* xchg is implicitly locked */ > ret > > /* initial value is 0xff, or "busy" */ > .globl bootlock >bootlock: .byte 0xff > >Of course, there would be nothing stopping us using this simple lock >primative to protect a counting semaphore that allowed recursion and >didn't have to worry about atomic code. > >Then there's the 'lock ; bts ...' (bit test and set) and so on >instructions. > >4.4Lite2 has a lock 'manager' as well, that uses a simple lock to protect >the complex locking structures. There is a good description of how this >all fits together in a Lite2 context somewhere on the web. I think this >is on Steve's pages, if not I'll have to check my bookmarks. I dont think I have seen this there. Could you post the url? >There are other lock managers around, I believe Terry has mentioned >one or two tree-based managers as well - these can be used to >control the deadlock risks etc. Intention locking is one. There are a few good posts in the mail archives about that. It is also on freebsd-smp page as well.. >Having a single lock is nice and convenient at this stage of the game, >since we do not have to worry so much about kernel deadlocks while we >figure out the fundamental core functionality. (eg: APIC_IO, IPI's, >TLB sync, scheduling, starup/shutdown, idle loops, NCPU > 2 support etc) I wasn't referring to a fine grained implementation right off hand. Just a couple more locks, and it would be quite difficult to get into a deadlock situation with them. Even so, you could still call it giant locking.. :P Chris > >Cheers, >-Peter From owner-freebsd-smp Thu Dec 5 08:53:07 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id IAA18157 for smp-outgoing; Thu, 5 Dec 1996 08:53:07 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id IAA18151 for ; Thu, 5 Dec 1996 08:53:00 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id JAA13340; Thu, 5 Dec 1996 09:51:49 -0700 Message-Id: <199612051651.JAA13340@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: "Erich Boleyn,,,," cc: Terje Normann Marthinussen , smp@freebsd.org Subject: Re: >2 CPU SMP test In-reply-to: Your message of "Thu, 05 Dec 1996 08:22:06 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 09:51:49 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > What part of the CMOS memory do you use for the SMP boot test? > > Erich Boleyn > 0x34-0x39, my BIOS book shows 0x34-0x3f as "reserved" Terje just reported: check: 99, 99, 99, 99, 99, 99 check: 1, 2, 3, 4, 5, 99 < AP 1 ran OK check: 99, 99, 99, 99, 99, 99 Application Processor #3 failed! ^ PHY# == LOG 2 check: 99, 99, 99, 99, 99, 99 < AP 2 failed to even start check: 99, 99, 99, 99, 99, 99 Application Processor #4 failed! ^ PHY# == LOG 3 check: 99, 99, 99, 99, 99, 99 < AP 3 failed to even start --- the very fisrt instructions run by the AP are: NON_GPROF_ENTRY(bootMP) cli CHECK(0x34,1) we see that the 1st AP had no problem setting it. but we see the initial 99 still in that checkpoint for the 2nd/3rd APs. from a previous test I had Terje run we proved that AP 3 (ie the last AP) can run by reversing the order of the loop calling startAP(). In that test AP3 ran, AP2 and AP1 failed. So its NOT specific CPUs, its launching more than 1 AP that breaks. some possible reasons: the bootstrap code, bootMP & MPentry, gets corrupted by being run the 1st time. the code in startAP() somehow leaves its APIC registers in a state where a second run of startAP() doesn't function the same. the first run of startAP() somehow affects all APs, leaving them in a state where they can't respond properly. (partially INITed, or ?) ??? one good test of several of the above would be to modify startAllAPs()" /* install the AP 1st level boot code */ installApTramp( bootAddr ); /* save the current value of the warm-start vector */ mpbioswarmvec = *((u_long*)WARMBOOT_OFF); outb( CMOS_REG, BIOS_RESET ); mpbiosreason = inb( CMOS_DATA ); /* setup a vector to our boot code */ *((volatile u_short *)WARMBOOT_OFF) = WARMBOOT_TARGET; *((volatile u_short *)WARMBOOT_SEG) = (bootAddr >> 4); outb( CMOS_REG, BIOS_RESET ); outb( CMOS_DATA, BIOS_WARM ); /* write 'warm-start' */ /* start each AP */ for ( x = 1; x <= mp_naps; ++x ) { startAP( CPU_TO_ID( x ), (bootAddr >> 12) & 0xff ); cpuApicVersions[ x ] = cpuApicVersions[ 0 ]; } to: /* save the current value of the warm-start vector */ mpbioswarmvec = *((u_long*)WARMBOOT_OFF); outb( CMOS_REG, BIOS_RESET ); mpbiosreason = inb( CMOS_DATA ); /* start each AP */ for ( x = 1; x <= mp_naps; ++x ) { /* install the AP 1st level boot code */ installApTramp( bootAddr ); /* setup a vector to our boot code */ *((volatile u_short *)WARMBOOT_OFF) = WARMBOOT_TARGET; *((volatile u_short *)WARMBOOT_SEG) = (bootAddr >> 4); outb( CMOS_REG, BIOS_RESET ); outb( CMOS_DATA, BIOS_WARM ); /* write 'warm-start' */ startAP( CPU_TO_ID( x ), (bootAddr >> 12) & 0xff ); cpuApicVersions[ x ] = cpuApicVersions[ 0 ]; } -- what we are doing here is reloading the trampoline code each pass and reloading the WARM BOOT vector incase the short "warm boot" section of BIOS run by the APs is changing that address/reason out from under us. for the first test I would NOT do the installApTramp() each time, its the least likely of the 2 to be wrong, and I'm not sure if it will 'install' cleanly 2 times in a row as it is modified in place for runtime addreses. It probably will re-install, but I would have to review to be sure. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 09:24:22 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA19515 for smp-outgoing; Thu, 5 Dec 1996 09:24:22 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id JAA19508 for ; Thu, 5 Dec 1996 09:24:17 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA13519; Thu, 5 Dec 1996 10:23:56 -0700 Message-Id: <199612051723.KAA13519@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Chris Csanady cc: Peter Wemm , smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 08:46:59 CST." <199612051447.IAA04684@friley216.res.iastate.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 10:23:56 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > ... >I wasn't referring to a fine grained implementation right off hand. Just >a couple more locks, and it would be quite difficult to get into a deadlock >situation with them. Even so, you could still call it giant locking.. :P another issue related to the current "giant lock" scheme is INT steering. The APICs use a 'Task Priority Register' and 'Process Priority Register' to determine which CPU gets IRQs from the IO APIC. Basically, the CPU running at the LOWEST priority gets sent any pending IRQs. The problem with the "giant lock" is that when a CPU holds it, no other CPU can service an INT since it spins on the lock. So whenever a CPU acquires the lock I have the code set the priority of that CPU to "VERY LOW". So now all pending INTs will be sent to it, which it can deal with since it already holds the lock. When a CPU releaes the lock it resets its priority to "NORMAL". The problem is that whenever a CPU gets an INT its PPR is increased to the priority of the INT, thus negating the "VERY LOW" prio I set in the TPR. So if a CPU is behind the lock because of an INT it WON'T get sent other pending INTs, the other CPU will. Now that we have IPIs working I hope to be able to send an IPI to the other CPUs telling them to set their TPR to "VERY HIGH", thus dis-abling them from getting INTs. After reading the last paragraph you might wonder why I even bother setting the CPU TPR to "VERY LOW", its because it is still beneficial for all other cases where the CPU holds the lock, syscalls, etc. In these cases the CPU PPR is not set by a pending INT and the TPR is the sole arbitor of the INT steerage. Anyways, the bottom line is that finer grain locking has to consider this INT streerage situation as well as the million other details involved! -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 09:45:35 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA20347 for smp-outgoing; Thu, 5 Dec 1996 09:45:35 -0800 (PST) Received: from cs.utah.edu (cs.utah.edu [128.110.4.21]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id JAA20342 for ; Thu, 5 Dec 1996 09:45:32 -0800 (PST) Received: from fast.cs.utah.edu by cs.utah.edu (8.8.3/utah-2.21-cs) id KAA18200; Thu, 5 Dec 1996 10:45:11 -0700 (MST) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id KAA18517; Thu, 5 Dec 1996 10:45:07 -0700 Date: Thu, 5 Dec 1996 10:45:07 -0700 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199612051745.KAA18517@fast.cs.utah.edu> To: ccsanady@friley216.res.iastate.edu, smp@csn.net Subject: Re: make locking more generic? Cc: peter@spinner.dialix.com, smp@freebsd.org Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Yes, the reason you need finer grained locking is because the interrupts *should* go to the other processor. If one processor is handling an interrupt and annother int comes in, the other CPU should be able to handle it. This would finally give parallel I/O! Linux doesn't do this, and they do very poorly when not every process is CPU bound. Kevin ps: This will most likely mean fixing device drivers as well. From owner-freebsd-smp Thu Dec 5 09:49:09 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA20474 for smp-outgoing; Thu, 5 Dec 1996 09:49:09 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id JAA20469 for ; Thu, 5 Dec 1996 09:49:06 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA13670; Thu, 5 Dec 1996 10:48:30 -0700 Message-Id: <199612051748.KAA13670@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 10:45:07 MST." <199612051745.KAA18517@fast.cs.utah.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 10:48:30 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Yes, the reason you need finer grained locking is because the > interrupts *should* go to the other processor. If one > processor is handling an interrupt and annother int comes > in, the other CPU should be able to handle it. This > would finally give parallel I/O! Linux doesn't do this, > and they do very poorly when not every process is CPU bound. > > Kevin > > ps: This will most likely mean fixing device drivers as well. what is your thinking here, ie what way are they likely to break? -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 09:59:22 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA20843 for smp-outgoing; Thu, 5 Dec 1996 09:59:22 -0800 (PST) Received: from cs.utah.edu (cs.utah.edu [128.110.4.21]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id JAA20838 for ; Thu, 5 Dec 1996 09:59:17 -0800 (PST) Received: from fast.cs.utah.edu by cs.utah.edu (8.8.3/utah-2.21-cs) id KAA18537; Thu, 5 Dec 1996 10:59:07 -0700 (MST) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id KAA19283; Thu, 5 Dec 1996 10:59:01 -0700 Date: Thu, 5 Dec 1996 10:59:01 -0700 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199612051759.KAA19283@fast.cs.utah.edu> To: smp@csn.net, vanmaren@fast.cs.utah.edu Subject: Re: make locking more generic? Cc: ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk My thinking was mainly that the drivers have never been tested in this environment. If you have a driver that is used for two devices, and you allow both to generate interrupts and have the driver executing on two processors simultaneously, the shared data structures are not going to be protected. So the simple fix would be to put a lock around each driver. But you will still have problems with other shared data structures; the same problem with allowing multiple processes to make kernel calls. This is all part of the enormous problem of adding fine-grained locking to an existing single-threaded kernel. Kevin From owner-freebsd-smp Thu Dec 5 10:04:50 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA21085 for smp-outgoing; Thu, 5 Dec 1996 10:04:50 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id KAA21080 for ; Thu, 5 Dec 1996 10:04:46 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id LAA13790; Thu, 5 Dec 1996 11:04:17 -0700 Message-Id: <199612051804.LAA13790@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 10:59:01 MST." <199612051759.KAA19283@fast.cs.utah.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 11:04:17 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > If you have a driver that is used for two devices, and you > allow both to generate interrupts and have the driver executing > on two processors simultaneously, the shared data structures > are not going to be protected. So the simple fix would be to > put a lock around each driver. But you will still have > problems with other shared data structures; the same problem > with allowing multiple processes to make kernel calls. one thing to know about is that the APIC has a notion of 'focus processor'. this means that if a CPU is currently servicing a specific IRQ, if that IRQ is generated again the request is sent to that CPU, reguardless of TPR and/or PPR > This is all part of the enormous problem of adding fine-grained > locking to an existing single-threaded kernel. yes, it will be a HUGE task to get it right! -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 10:34:56 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA22690 for smp-outgoing; Thu, 5 Dec 1996 10:34:56 -0800 (PST) Received: from dyson.iquest.net (dyson.iquest.net [198.70.144.127]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id KAA22683 for ; Thu, 5 Dec 1996 10:34:53 -0800 (PST) Received: (from root@localhost) by dyson.iquest.net (8.8.2/8.6.9) id NAA05328; Thu, 5 Dec 1996 13:34:01 -0500 (EST) From: John Dyson Message-Id: <199612051834.NAA05328@dyson.iquest.net> Subject: Re: make locking more generic? To: smp@csn.net (Steve Passe) Date: Thu, 5 Dec 1996 13:34:01 -0500 (EST) Cc: vanmaren@fast.cs.utah.edu, ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org In-Reply-To: <199612051804.LAA13790@clem.systemsix.com> from "Steve Passe" at Dec 5, 96 11:04:17 am Reply-To: dyson@freebsd.org X-Mailer: ELM [version 2.4 PL24 ME8] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > This is all part of the enormous problem of adding fine-grained > > locking to an existing single-threaded kernel. > > yes, it will be a HUGE task to get it right! > That is the reason that I am kind-of asking which directions that you are looking at. If you don't have anything in mind at the VFS/VM levels, I will look at it from a reasonable viewpoint myself... John From owner-freebsd-smp Thu Dec 5 10:58:23 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA24290 for smp-outgoing; Thu, 5 Dec 1996 10:58:23 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id KAA24282 for ; Thu, 5 Dec 1996 10:58:17 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id LAA19621; Thu, 5 Dec 1996 11:37:58 -0700 From: Terry Lambert Message-Id: <199612051837.LAA19621@phaeton.artisoft.com> Subject: Re: make locking more generic? To: smp@csn.net (Steve Passe) Date: Thu, 5 Dec 1996 11:37:58 -0700 (MST) Cc: ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org In-Reply-To: <199612051723.KAA13519@clem.systemsix.com> from "Steve Passe" at Dec 5, 96 10:23:56 am X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > So whenever a CPU acquires the lock I have > the code set the priority of that CPU to "VERY LOW". So now all pending INTs > will be sent to it, which it can deal with since it already holds the lock. > When a CPU releaes the lock it resets its priority to "NORMAL". > > The problem is that whenever a CPU gets an INT its PPR is increased to > the priority of the INT, thus negating the "VERY LOW" prio I set in the > TPR. So if a CPU is behind the lock because of an INT it WON'T get > sent other pending INTs, the other CPU will. Now that we have IPIs > working I hope to be able to send an IPI to the other CPUs telling them to set > their TPR to "VERY HIGH", thus dis-abling them from getting INTs. I don't understand. If CPU 1 is servicing INT 12 from a disk controller, and you get INT 4 from a serial port, don't you *want* CPU 2 to service the INT 4 in parallel??? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Thu Dec 5 11:03:38 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA24645 for smp-outgoing; Thu, 5 Dec 1996 11:03:38 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA24631 for ; Thu, 5 Dec 1996 11:03:33 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id LAA19641; Thu, 5 Dec 1996 11:42:39 -0700 From: Terry Lambert Message-Id: <199612051842.LAA19641@phaeton.artisoft.com> Subject: Re: make locking more generic? To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Date: Thu, 5 Dec 1996 11:42:39 -0700 (MST) Cc: smp@csn.net, vanmaren@fast.cs.utah.edu, ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org In-Reply-To: <199612051759.KAA19283@fast.cs.utah.edu> from "Kevin Van Maren" at Dec 5, 96 10:59:01 am X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > If you have a driver that is used for two devices, and you > allow both to generate interrupts and have the driver executing > on two processors simultaneously, the shared data structures > are not going to be protected. So the simple fix would be to > put a lock around each driver. But you will still have > problems with other shared data structures; the same problem > with allowing multiple processes to make kernel calls. I believe the correct intermediate method would be to bind the driver service to the CPU for the duration of the service cycle, such that other interrupts for the driver are serviced by the same CPU. This is somewhat counter-intuitive; you might think that a lock around the driver is the simple fix; but it affords no migration path for getting concurrency before you get loose CPU binding. So you want: 1) The current situation 2) Driver/CPU affinity 3) Driver/CPU independency My opinion, anyway. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Thu Dec 5 11:05:58 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA24793 for smp-outgoing; Thu, 5 Dec 1996 11:05:58 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id LAA24783 for ; Thu, 5 Dec 1996 11:05:48 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id UAA01282; Thu, 5 Dec 1996 20:05:35 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id UAA20174; Thu, 5 Dec 1996 20:05:34 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id UAA20413; Thu, 5 Dec 1996 20:05:33 +0100 (MET) Message-Id: <199612051905.UAA20413@slibo.cc.uit.no> To: Steve Passe cc: "Erich Boleyn,,,," , smp@freebsd.org Subject: Re: >2 CPU SMP test In-reply-to: Your message of "Thu, 05 Dec 1996 09:51:49 MET." <199612051651.JAA13340@clem.systemsix.com> Date: Thu, 05 Dec 1996 20:05:32 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk TADA!!!!!!!!!!!! :) I did: for ( x = 1; x <= mp_naps; ++x ) { /* setup a vector to our boot code */ *((volatile u_short *)WARMBOOT_OFF) = WARMBOOT_TARGET; *((volatile u_short *)WARMBOOT_SEG) = (bootAddr >> 4); outb( CMOS_REG, BIOS_RESET ); outb( CMOS_DATA, BIOS_WARM ); /* write 'warm-start' */ startAP( CPU_TO_ID( x ), (bootAddr >> 12) & 0xff ); cpuApicVersions[ x ] = cpuApicVersions[ 0 ]; } Like suggested, and: check: 99, 99, 99, 99, 99, 99 check: 1, 2, 3, 4, 5, 99 check: 99, 99, 99, 99, 99, 99 check: 1, 2, 3, 4, 5, 99 check: 99, 99, 99, 99, 99, 99 check: 1, 2, 3, 4, 5, 99 ... # sysctl -w kern.smp_active=4 SMP: Starting 1st AP! SMP: AP CPU #2 LAUNCHED!! Starting Scheduling... SMP: TADA! CPU #2 made it into the scheduler!. SMP: 2 of 4 CPU's online. Unlocking next CPU.. SMP: AP CPU #3 LAUNCHED!! Starting Scheduling... SMP: TADA! CPU #3 made it into the scheduler!. SMP: 3 of 4 CPU's online. Unlocking next CPU.. SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... SMP: TADA! CPU #1 made it into the scheduler!. SMP: All 4 CPU's are online! And of course, some kernel makes: time make 209.2u 140.7s 6:44.98 86.4% 228+262k 69+5332io 0pf+0w time make -j 2 131.4u 230.1s 4:20.98 138.5% 54+58k 69+7071io 0pf+0w time make -j 4 210.8u 165.7s 2:34.21 244.1% 126+157k 70+7288io 0pf+0w time make -j 7 233.7u 149.5s 2:08.82 297.5% 138+182k 82+7534io 14pf+0w time make -j 14 238.2u 145.8s 2:04.22 309.2% 131+179k 75+7280io 0pf+0w time make -j 16 249.5u 136.0s 2:07.73 301.8% 136+177k 78+7324io 0pf+0w And then I remembered that I didn't use -pipe (sorry, done with shell script, different shell, differen time output, didn't care to do it again): in miutes compared to j=1 time make 342.12 real 256.07 user 111.05 sys 5:42.12 1 time make -j 2 226.33 real 268.54 user 117.25 sys 3:46.33 1.51 time make -j 4 126.64 real 257.72 user 130.47 sys 2:06.64 2.70 time make -j 7 108.19 real 271.30 user 114.41 sys 1:48.19 3.16 time make -j 14 107.22 real 274.56 user 114.16 sys 1:47.22 3.19 time make -j 14 with csh reported 361.3% cpu usage. Not tried to much yet, but the interactive response seems good at all times. Think I'll have to take the disk up a couple of floors and try it on some 4CPU P6 netservers there :) Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Thu Dec 5 11:06:53 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA24838 for smp-outgoing; Thu, 5 Dec 1996 11:06:53 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA24827 for ; Thu, 5 Dec 1996 11:06:51 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id LAA19654; Thu, 5 Dec 1996 11:46:00 -0700 From: Terry Lambert Message-Id: <199612051846.LAA19654@phaeton.artisoft.com> Subject: Re: make locking more generic? To: smp@csn.net (Steve Passe) Date: Thu, 5 Dec 1996 11:45:59 -0700 (MST) Cc: vanmaren@fast.cs.utah.edu, ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org In-Reply-To: <199612051804.LAA13790@clem.systemsix.com> from "Steve Passe" at Dec 5, 96 11:04:17 am X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > If you have a driver that is used for two devices, and you > > allow both to generate interrupts and have the driver executing > > on two processors simultaneously, the shared data structures > > are not going to be protected. So the simple fix would be to > > put a lock around each driver. But you will still have > > problems with other shared data structures; the same problem > > with allowing multiple processes to make kernel calls. > > one thing to know about is that the APIC has a notion of 'focus processor'. > this means that if a CPU is currently servicing a specific IRQ, if that > IRQ is generated again the request is sent to that CPU, reguardless of > TPR and/or PPR I think Kevin was more conserned about having a single Adaptec driver and two controllers, on different interrupts, but with perhaps serially accessed global structures. The serial access is the result of the way interrupt delivery is currently handled. For instance, I may get INT 14 and INT 15 to enter the same driver, which has an unguarded list for, for instance, available bounce buffers, regardless of the number of controllers. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Thu Dec 5 11:18:29 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA25498 for smp-outgoing; Thu, 5 Dec 1996 11:18:29 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA25488 for ; Thu, 5 Dec 1996 11:18:24 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVjN4-0005HN-00; Thu, 5 Dec 1996 11:22:10 -0800 To: Steve Passe cc: Terje Normann Marthinussen , smp@freebsd.org Subject: Re: >2 CPU SMP test In-reply-to: Your message of "Thu, 05 Dec 1996 09:51:49 MST." <199612051651.JAA13340@clem.systemsix.com> Date: Thu, 05 Dec 1996 11:22:10 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve Passe writes: ... > some possible reasons: > > the bootstrap code, bootMP & MPentry, gets corrupted by being run the 1st > time. > > the code in startAP() somehow leaves its APIC registers in a state where > a second run of startAP() doesn't function the same. > > the first run of startAP() somehow affects all APs, leaving them in a state > where they can't respond properly. (partially INITed, or ?) > > ??? > > one good test of several of the above would be to modify startAllAPs()" > > /* install the AP 1st level boot code */ > installApTramp( bootAddr ); > > /* save the current value of the warm-start vector */ > mpbioswarmvec = *((u_long*)WARMBOOT_OFF); > outb( CMOS_REG, BIOS_RESET ); > mpbiosreason = inb( CMOS_DATA ); > > /* setup a vector to our boot code */ > *((volatile u_short *)WARMBOOT_OFF) = WARMBOOT_TARGET; > *((volatile u_short *)WARMBOOT_SEG) = (bootAddr >> 4); > outb( CMOS_REG, BIOS_RESET ); > outb( CMOS_DATA, BIOS_WARM ); /* write 'warm-start' */ > > /* start each AP */ > for ( x = 1; x <= mp_naps; ++x ) { > startAP( CPU_TO_ID( x ), (bootAddr >> 12) & 0xff ); > cpuApicVersions[ x ] = cpuApicVersions[ 0 ]; > } > > to: ... Your problem here is definitely that the warm reset code and vector are not being set for each CPU. The *entire* setup sequence for AP CPU bootup must be done for each CPU (except maybe a memcpy that isn't overwritten by anything the AP CPU does). In particular, on an XXPRESS-based Pentium system, the STARTUP IPI is ignored, and it needs the warm reset code and vector. The problem here is that the reset code is changed to indicate what happened to the CPU after it was reset. For a Pentium Pro-based system, your code would work correctly, since all of them that I've ever heard of use the integrated APIC stuff correctly (and therefore use the STARTUP IPI). -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Thu Dec 5 11:43:23 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA27594 for smp-outgoing; Thu, 5 Dec 1996 11:43:23 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA27588 for ; Thu, 5 Dec 1996 11:43:19 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id MAA14434; Thu, 5 Dec 1996 12:40:56 -0700 Message-Id: <199612051940.MAA14434@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 11:37:58 MST." <199612051837.LAA19621@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 12:40:55 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >> So whenever a CPU acquires the lock I have >> the code set the priority of that CPU to "VERY LOW". So now all pending INTs >> will be sent to it, which it can deal with since it already holds the lock. >> When a CPU releaes the lock it resets its priority to "NORMAL". >> >> The problem is that whenever a CPU gets an INT its PPR is increased to >> the priority of the INT, thus negating the "VERY LOW" prio I set in the >> TPR. So if a CPU is behind the lock because of an INT it WON'T get >> sent other pending INTs, the other CPU will. Now that we have IPIs .> working I hope to be able to send an IPI to the other CPUs telling them to set >> their TPR to "VERY HIGH", thus dis-abling them from getting INTs. > >I don't understand. > >If CPU 1 is servicing INT 12 from a disk controller, and you get INT 4 >from a serial port, don't you *want* CPU 2 to service the INT 4 in >parallel??? yes, you do. but the problem is that CPU 1 holds the lock, so if CPU2 tries to service INT 4, it starts spinning in the un-obtainable lock till CPU1 finishes servicing INT 12. At this point CPU1 releases the lock, CPU2 finally gets it and can proceed to service INT 4. Might as well let CPU2 alone to do whatever it is doing instead. If the INT 4 is of higher priority than INT 12, CPU1 could start servicing it immediatly. The proper form of fine-grainularity could eliminate this problem. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 11:56:32 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA28309 for smp-outgoing; Thu, 5 Dec 1996 11:56:32 -0800 (PST) Received: from INET-02-IMC.microsoft.com (mail2.microsoft.com [131.107.3.42]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA28302 for ; Thu, 5 Dec 1996 11:56:30 -0800 (PST) Received: by INET-02-IMC.microsoft.com with SMTP (Microsoft Exchange Server Internet Mail Connector Version 4.0.994.63) id <01BBE29A.E3253CD0@INET-02-IMC.microsoft.com>; Thu, 5 Dec 1996 10:55:54 -0800 Message-ID: From: Thomas Pfenning To: "'Chris Csanady'" , "'Peter Wemm'" Cc: "'smp@freebsd.org'" Subject: RE: make locking more generic? Date: Thu, 5 Dec 1996 10:55:33 -0800 X-Mailer: Microsoft Exchange Server Internet Mail Connector Version 4.0.994.63 Encoding: 100 TEXT Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Doesn't the lock version in this mail actual trash the cached value for bootlock on every spin? What about using MCS queueing locks to solve both the cache trashing and the reentrance at the same time. Cheers Thomas >-----Original Message----- >From: Chris Csanady [SMTP:ccsanady@friley216.res.iastate.edu] >Sent: Thursday, December 05, 1996 6:47 AM >To: Peter Wemm >Cc: smp@freebsd.org >Subject: Re: make locking more generic? > > >>Steve Passe wrote: >>> Hi, >>> >>> >If any of you SMP guys have any words of wisdom, or documentation sources >>> >for SMP technology, I would really like to hear from you with pointers. >>>Al >s >> o, >>> >>> http://www.freebsd.org/~fsmp/SMP/books.html >>> http://www.freebsd.org/~fsmp/SMP/threads.html >>> -- >>> Steve Passe | powered by >>> smp@csn.net | FreeBSD >> >>There are also several "classes" of locks to consider. The version >>that we use to guard the entry/exit points of the kernel allows recursion >>and hence has a "nest count" which accounts for it's complexity and it's >>impact on the task switch code. By recursion, I mean that a process >>can enter the kernel lock, call something which ends up in copyin() >>and takes another fault, which re-enters the kernel lock again. it needs >>to keep track of how many times each process has entered the kernel so >>that it can unwind it correctly. >> >>Then there's a simple boolean lock. It's a simple request/free lock >>that does not care which cpu is in it at the time and does not allow >>recursion. >> >>ENTRY(boot_lock) >> /* This is the Intel reccomended semaphore method */ >> movb $0xff, %al >>2: >> xchgb %al, bootlock /* xchg is implicitly locked */ >> cmpb $0xff, %al >> jz 2b >> ret >> >>ENTRY(boot_unlock) >> movb $0, %al >> xchgb %al, bootlock /* xchg is implicitly locked */ >> ret >> >> /* initial value is 0xff, or "busy" */ >> .globl bootlock >>bootlock: .byte 0xff >> >>Of course, there would be nothing stopping us using this simple lock >>primative to protect a counting semaphore that allowed recursion and >>didn't have to worry about atomic code. >> >>Then there's the 'lock ; bts ...' (bit test and set) and so on >>instructions. >> >>4.4Lite2 has a lock 'manager' as well, that uses a simple lock to protect >>the complex locking structures. There is a good description of how this >>all fits together in a Lite2 context somewhere on the web. I think this >>is on Steve's pages, if not I'll have to check my bookmarks. > >I dont think I have seen this there. Could you post the url? > >>There are other lock managers around, I believe Terry has mentioned >>one or two tree-based managers as well - these can be used to >>control the deadlock risks etc. > >Intention locking is one. There are a few good posts in the mail archives >about that. It is also on freebsd-smp page as well.. > >>Having a single lock is nice and convenient at this stage of the game, >>since we do not have to worry so much about kernel deadlocks while we >>figure out the fundamental core functionality. (eg: APIC_IO, IPI's, >>TLB sync, scheduling, starup/shutdown, idle loops, NCPU > 2 support etc) > >I wasn't referring to a fine grained implementation right off hand. Just >a couple more locks, and it would be quite difficult to get into a deadlock >situation with them. Even so, you could still call it giant locking.. :P > >Chris > >> >>Cheers, >>-Peter > > > From owner-freebsd-smp Thu Dec 5 12:21:40 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id MAA01213 for smp-outgoing; Thu, 5 Dec 1996 12:21:40 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id MAA01206 for ; Thu, 5 Dec 1996 12:21:38 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id NAA19785; Thu, 5 Dec 1996 13:01:18 -0700 From: Terry Lambert Message-Id: <199612052001.NAA19785@phaeton.artisoft.com> Subject: Re: make locking more generic? To: smp@csn.net (Steve Passe) Date: Thu, 5 Dec 1996 13:01:17 -0700 (MST) Cc: terry@lambert.org, ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org In-Reply-To: <199612051940.MAA14434@clem.systemsix.com> from "Steve Passe" at Dec 5, 96 12:40:55 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > >If CPU 1 is servicing INT 12 from a disk controller, and you get INT 4 > >from a serial port, don't you *want* CPU 2 to service the INT 4 in > >parallel??? > > yes, you do. but the problem is that CPU 1 holds the lock, so if CPU2 > tries to service INT 4, it starts spinning in the un-obtainable lock till CPU1 > finishes servicing INT 12. At this point CPU1 releases the lock, CPU2 > finally gets it and can proceed to service INT 4. Might as well let CPU2 > alone to do whatever it is doing instead. If the INT 4 is of higher priority > than INT 12, CPU1 could start servicing it immediatly. > > The proper form of fine-grainularity could eliminate this problem. Ah. I see. We are still talking about the global kernel entrancy lock in this case. Never mind then (the global entrancy lock is bogus). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Thu Dec 5 12:43:11 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id MAA02417 for smp-outgoing; Thu, 5 Dec 1996 12:43:11 -0800 (PST) Received: from kvikk.uit.no (kvikk.Uit.No [129.242.4.32]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id MAA02412 for ; Thu, 5 Dec 1996 12:43:07 -0800 (PST) Received: from sprint.cc.uit.no (sprint.Cc.Uit.No [129.242.5.198]) by kvikk.uit.no (8.7.3/8.7.1) with ESMTP id VAA02468; Thu, 5 Dec 1996 21:42:58 +0100 (MET) Received: from slibo.cc.uit.no (slibo.Cc.Uit.No [129.242.5.36]) by sprint.cc.uit.no (8.8.0/8.8.0) with ESMTP id VAA20636; Thu, 5 Dec 1996 21:42:57 +0100 (MET) Received: from localhost (terjem@localhost) by slibo.cc.uit.no (8.7.3/8.7.3) with ESMTP id VAA22871; Thu, 5 Dec 1996 21:42:57 +0100 (MET) Message-Id: <199612052042.VAA22871@slibo.cc.uit.no> To: smp@freebsd.org cc: Steve Passe , "Erich Boleyn,,,," Subject: Re: >2 CPU SMP test In-reply-to: Your message of "Thu, 05 Dec 1996 20:05:32 MET." <199612051905.UAA20413@slibo.cc.uit.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 21:42:56 +0100 From: Terje Normann Marthinussen Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >TADA!!!!!!!!!!!! :) > >I did: > for ( x = 1; x <= mp_naps; ++x ) { > /* setup a vector to our boot code */ > *((volatile u_short *)WARMBOOT_OFF) = WARMBOOT_TARGET; > *((volatile u_short *)WARMBOOT_SEG) = (bootAddr >> 4); > outb( CMOS_REG, BIOS_RESET ); > outb( CMOS_DATA, BIOS_WARM ); /* write 'warm-start' */ > > startAP( CPU_TO_ID( x ), (bootAddr >> 12) & 0xff ); > cpuApicVersions[ x ] = cpuApicVersions[ 0 ]; > } Seems like there was an additional bonus here. It also made APIC_IO work. Just done a bunch of kernel compilations with both APIC_IO and SMP_INVLTLB as well as all 4 CPU's running. Haven't noticed any problems yet. I've tried some patches from Steve lately, as well as some small changes myself, so I did a fresh cvs checkout of the entire smp-current and it looks like the fix above is all that is needed. Terje Marthinussen terjem@cc.uit.no From owner-freebsd-smp Thu Dec 5 12:49:00 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id MAA02558 for smp-outgoing; Thu, 5 Dec 1996 12:49:00 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id MAA02553 for ; Thu, 5 Dec 1996 12:48:55 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id NAA14783; Thu, 5 Dec 1996 13:48:43 -0700 Message-Id: <199612052048.NAA14783@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terje Normann Marthinussen cc: smp@freebsd.org, "Erich Boleyn,,,," Subject: Re: >2 CPU SMP test In-reply-to: Your message of "Thu, 05 Dec 1996 21:42:56 +0100." <199612052042.VAA22871@slibo.cc.uit.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 13:48:42 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > >TADA!!!!!!!!!!!! :) > ... > Seems like there was an additional bonus here. > > It also made APIC_IO work. Just done a bunch of kernel compilations > with both APIC_IO and SMP_INVLTLB as well as all 4 CPU's running. Haven't > noticed any problems yet. > > I've tried some patches from Steve lately, as well as some small changes > myself, so I did a fresh cvs checkout of the entire smp-current and it > looks like the fix above is all that is needed. I am cleaning up the CHECK() macros and making them part of the code, they proved quite useful for finding this bug. I am alos cleaning up the code in startAllAPs() and startAP() to make the messages, etc clearer. Should be checking it in in a few hours, want to test-compile etc first. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 14:31:19 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id OAA09990 for smp-outgoing; Thu, 5 Dec 1996 14:31:19 -0800 (PST) Received: from pat.idt.unit.no (0@pat.idt.unit.no [129.241.103.5]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id OAA09976 for ; Thu, 5 Dec 1996 14:31:12 -0800 (PST) Received: from idt.unit.no (tegge@ikke.idt.unit.no [129.241.111.65]) by pat.idt.unit.no (8.8.3/8.8.3) with ESMTP id XAA03136; Thu, 5 Dec 1996 23:30:35 +0100 (MET) Message-Id: <199612052230.XAA03136@pat.idt.unit.no> To: smp@csn.net Cc: vanmaren@fast.cs.utah.edu, smp@bluenose.na.tuns.ca, smp@freebsd.org Subject: Re: Good News & Bad News In-Reply-To: Your message of "Wed, 04 Dec 1996 16:40:45 -0700" References: <199612042340.QAA08544@clem.systemsix.com> X-Mailer: Mew version 1.06 on Emacs 19.33.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Thu, 05 Dec 1996 23:30:34 +0100 From: Tor Egge Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Hi, > > > >With the current smp kernel+ SMP_INVLTLB, the good news is that Tomcat III+IDE > > >is able to `make world' successfully with FreeBSD-current source tree. > > > > > >With the same kernel, the Titan Pro is still suffering with > > >the core dump of using IDE right after the second CPU is launched. > > >If the SCSI is used to boot the system, there is no coredump but > > >the system reboots itself unpredictably shortly after the second > > >CPU is activated. > > > > Just a thought, but how many of the machines besides the Tyan Titan > > Pro have the BSP at other than CPU #0? The Titan Pro's BSD is > > (physically) Processor #1. This is just the most obvious difference > > between the Titan Pro and the other boards. > > from the mptab;e database: > > Asus P/I-P65UP5 > Supermicro P6DNE > > does anyone running these machines have experience with the recent SMP kernel? I tried a few hours old kernel on an ASUS P/I-P65UP5, with APIC_IO enabled. When compiling a new kernel with two active CPUs, I got error messages from gcc, and the compile failed. Restarting the kernel compiling caused a trap 12, and a kernel dump. When looking at the kernel dump, I get #0 boot (howto=256) at ../../kern/kern_shutdown.c:267 #1 0xe0112d29 in panic (fmt=0xe01bcbcf "page fault") at ../../kern/kern_shutdown.c:395 #2 0xe01bd8b5 in trap_fatal (frame=0xdfbffe58) at ../../i386/i386/trap.c:747 #3 0xe01bd2e8 in trap_pfault (frame=0xdfbffe58, usermode=0) at ../../i386/i386/trap.c:654 #4 0xe01bcf1b in trap (frame={tf_es = -270335984, tf_ds = 16, tf_edi = -270395292, tf_esi = -541077504, tf_ebp = -541065552, tf_isp = -541065600, tf_ebx = 296243200, tf_edx = -4194304, tf_ecx = -528396, tf_eax = -528396, tf_trapno = 12, tf_err = 0, tf_eip = -535058833, tf_cs = 8, tf_eflags = 66178, tf_esp = -530083069, tf_ss = -270296448}) at ../../i386/i386/trap.c:313 #5 0xe01ba66f in pmap_enter (pmap=0xefe21864, va=3753889792, pa=296243200, prot=7 '\a', wired=0) at ../../i386/i386/pmap.c:2014 #6 0xe01a4153 in vm_fault (map=0xefe21800, vaddr=3753889792, fault_type=3 '\003', change_wiring=0) at ../../vm/vm_fault.c:773 #7 0xe01bd240 in trap_pfault (frame=0xdfbfffbc, usermode=1) at ../../i386/i386/trap.c:634 #8 0xe01bcdc3 in trap (frame={tf_es = 39, tf_ds = 39, tf_edi = 352256, tf_esi = 330220, tf_ebp = -541074428, tf_isp = -541065244, tf_ebx = 0, tf_edx = 1, tf_ecx = 330220, tf_eax = 0, tf_trapno = 12, tf_err = 7, tf_eip = 45296, tf_cs = 31, tf_eflags = 66050, tf_esp = -541074452, tf_ss = 39}) at ../../i386/i386/trap.c:241 (kgdb) up 5 #5 0xe01ba66f in pmap_enter (pmap=0xefe21864, va=3753889792, pa=296243200, prot=7 '\a', wired=0) at ../../i386/i386/pmap.c:2014 2014 origpte = *(vm_offset_t *)pte; (kgdb) print/x pte $2 = 0xfff7eff4 This indicates an attempt to dereference address 0xfff7eff4, which seems bogus :-( - Tor Egge From owner-freebsd-smp Thu Dec 5 14:47:32 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id OAA10690 for smp-outgoing; Thu, 5 Dec 1996 14:47:32 -0800 (PST) Received: from cs.utah.edu (cs.utah.edu [128.110.4.21]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id OAA10684 for ; Thu, 5 Dec 1996 14:47:30 -0800 (PST) Received: from fast.cs.utah.edu by cs.utah.edu (8.8.3/utah-2.21-cs) id PAA25771; Thu, 5 Dec 1996 15:47:26 -0700 (MST) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id PAA29804; Thu, 5 Dec 1996 15:47:26 -0700 Date: Thu, 5 Dec 1996 15:47:26 -0700 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199612052247.PAA29804@fast.cs.utah.edu> To: smp@freebsd.org Subject: Re: make locking more generic? Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk We still have the problem with two (say, adaptec SCSI) devices on different IRQs vs two different devices on the same IRQ. This is why multiple I/O APICS would be great -- each device gets it own unique IRQ. Then the OS can play games redirecting IRQs for processor affinity interrupt handling, better load (IRQ) ballancing, etc. Kevin From owner-freebsd-smp Thu Dec 5 15:13:38 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id PAA11639 for smp-outgoing; Thu, 5 Dec 1996 15:13:38 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id PAA11630 for ; Thu, 5 Dec 1996 15:13:34 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id QAA15542; Thu, 5 Dec 1996 16:13:14 -0700 Message-Id: <199612052313.QAA15542@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 15:47:26 MST." <199612052247.PAA29804@fast.cs.utah.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 16:13:14 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > We still have the problem with two (say, adaptec SCSI) devices > on different IRQs vs two different devices on the same IRQ. > > This is why multiple I/O APICS would be great -- each device > gets it own unique IRQ. Then the OS can play games redirecting > IRQs for processor affinity interrupt handling, better load > (IRQ) ballancing, etc. multiple controllers can use unique PCI IRQs, IF the manufacturer designs the motherboard correctly, all on 1 IO APIC. Or the MB manufacturer can ignore those extra IRQs on the IO APIC and "redirect" PCI IRQs to traditional ISA IRQs. doing the kernel to account for all these "design considerations" that the various boards throw our way make it even more challenging! -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 15:29:07 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id PAA12152 for smp-outgoing; Thu, 5 Dec 1996 15:29:07 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id PAA12143 for ; Thu, 5 Dec 1996 15:29:03 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id QAA15632; Thu, 5 Dec 1996 16:28:19 -0700 Message-Id: <199612052328.QAA15632@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Tor Egge cc: smp@freebsd.org Subject: Re: Spontaneous reset In-reply-to: Your message of "Fri, 06 Dec 1996 00:24:21 +0100." <199612052324.AAA04330@pat.idt.unit.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 16:28:18 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > One interesting side effect of having played with a -SMP kernel is that > the machine must be turned off before booting a -current kernel. > > The -current kernel boots fine without the machine being turned off, > *BUT* resets spontaneously at 12:00 AM and 12:00 PM. I run > a cron job that kills two processes at those times, but under normal > circumstandes that does not cause a system reset. It only happens > after having played with the -SMP kernel. I'm not sure we properly halt the APs, they could still be running after a warm boot! I also suggest a power cycle whenever you hit one of the SCSI freakouts... -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 15:34:35 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id PAA12599 for smp-outgoing; Thu, 5 Dec 1996 15:34:35 -0800 (PST) Received: from hda.hda.com (ip32-max1-fitch.ziplink.net [199.232.245.32]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id PAA12560; Thu, 5 Dec 1996 15:34:19 -0800 (PST) Received: (from dufault@localhost) by hda.hda.com (8.6.12/8.6.12) id SAA29433; Thu, 5 Dec 1996 18:31:39 -0500 From: Peter Dufault Message-Id: <199612052331.SAA29433@hda.hda.com> Subject: Re: make locking more generic? In-Reply-To: <199612051834.NAA05328@dyson.iquest.net> from John Dyson at "Dec 5, 96 01:34:01 pm" To: dyson@freebsd.org Date: Thu, 5 Dec 1996 18:31:38 -0500 (EST) Cc: smp@csn.net, vanmaren@fast.cs.utah.edu, ccsanady@friley216.res.iastate.edu, peter@spinner.dialix.com, smp@freebsd.org X-Mailer: ELM [version 2.4ME+ PL25 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > > > This is all part of the enormous problem of adding fine-grained > > > locking to an existing single-threaded kernel. > > > > yes, it will be a HUGE task to get it right! > > > That is the reason that I am kind-of asking which directions that you > are looking at. If you don't have anything in mind at the VFS/VM > levels, I will look at it from a reasonable viewpoint myself... This doesn't relate to multi-threading the VM system, but from the viewpoint of VM support for locks it would be good to permit associating a region with a lock that could be unmapped or mapped read-only based on the state of an unlocked / read-only / read-write lock. Related to the earlier points about driver data structures and reentrancy: I believe the driver data structures themselves end up being independent based on the interrupt source and present less of a locking issue than the kernel support routines that the drivers call. Single threading the entrance into the top half of a driver per-controller as was suggested should get things off the ground. Sorry if I'm too obvious - I'm pretty new to following this list. -- Peter Dufault Real-Time Machine Control and Simulation HD Associates, Inc. Voice: 508 433 6936 dufault@hda.com Fax: 508 433 5267 From owner-freebsd-smp Thu Dec 5 15:38:15 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id PAA12722 for smp-outgoing; Thu, 5 Dec 1996 15:38:15 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id PAA12717 for ; Thu, 5 Dec 1996 15:38:13 -0800 (PST) Received: from cs.utah.edu (cs.utah.edu [128.110.4.21]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id PAA02719 for ; Thu, 5 Dec 1996 15:38:11 -0800 (PST) Received: from fast.cs.utah.edu by cs.utah.edu (8.8.3/utah-2.21-cs) id QAA26654; Thu, 5 Dec 1996 16:36:50 -0700 (MST) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id QAA00497; Thu, 5 Dec 1996 16:36:49 -0700 Date: Thu, 5 Dec 1996 16:36:49 -0700 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199612052336.QAA00497@fast.cs.utah.edu> To: smp@csn.net, vanmaren@fast.cs.utah.edu Subject: Re: make locking more generic? Cc: smp@freebsd.org Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk My main point is that you wouldn't be limited to 16 IRQ lines. From owner-freebsd-smp Thu Dec 5 15:39:12 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id PAA12743 for smp-outgoing; Thu, 5 Dec 1996 15:39:12 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id PAA12738 for ; Thu, 5 Dec 1996 15:39:08 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id QAA15728 for ; Thu, 5 Dec 1996 16:39:04 -0700 Message-Id: <199612052339.QAA15728@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@freebsd.org Subject: Re: Crashing on activating other CPUs In-reply-to: Your message of "Wed, 04 Dec 1996 10:33:24 MST." <199612041733.KAA06644@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 16:39:04 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > > Our 4 CPU Netserver works without SMP_INVLTLB and APIC_IO (the latter > > has never worked). With SMP_INVLTLB, I get something similar the > ... > I think the problem is that damn re-direction thingy for PCI INTs to ISA bus. > ... > your system says the PCI disk controler uses: > -- > I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# > INT conforms level 19 7 14 7 > ^^^^^^^^ ^^^^^ > > Which is different from the ISA standard of active-hi/edge. So far our > experiments suggested that for cards (PCI & EISA) redirected to ISA INTs > we should ignore the entries and stick with ISA levels. This works > on at least 1 EISA machine and 1 PCI machine that I have encountered. > However the printout you sent earlier shows that your PCI card is > completely loosing its INTs ubder APIC_IO, symptomatic of this pin > being incorrectly programmed. with Terje's success today running 4 CPUs with APIC_IO it appears I jumped to the wrong conclusion about this issue. So it appears to now be fact that: whenever PCI or EISA cards are redirected to ISA IRQs the levels get set to ISA values (active-hi/edge) reguardless of the "settings" in the mptable. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 16:02:51 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id QAA13595 for smp-outgoing; Thu, 5 Dec 1996 16:02:51 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id QAA13590 for ; Thu, 5 Dec 1996 16:02:46 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA15873; Thu, 5 Dec 1996 17:02:15 -0700 Message-Id: <199612060002.RAA15873@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: clintw@colorado.cirrus.com (Clint Wolff) cc: smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 16:32:52 MST." <9612052332.AA02586@longs.colorado.cirrus.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 17:02:15 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk hi, > > > We still have the problem with two (say, adaptec SCSI) devices > > > on different IRQs vs two different devices on the same IRQ. > > > > > > This is why multiple I/O APICS would be great -- each device > > > gets it own unique IRQ. Then the OS can play games redirecting > > > IRQs for processor affinity interrupt handling, better load > > > (IRQ) ballancing, etc. > > > > multiple controllers can use unique PCI IRQs, IF the manufacturer designs > > the motherboard correctly, all on 1 IO APIC. Or the MB manufacturer > > can ignore those extra IRQs on the IO APIC and "redirect" PCI IRQs > > to traditional ISA IRQs. doing the kernel to account for all these > > "design considerations" that the various boards throw our way make it > > even more challenging! > > > > > > Perhaps I am not understanding this all, but my SCSI controller (AHA-2940) > and my Ethernet controller (SMC EtherPower?? DEC PCI chipset) BOTH want > to be on INTA... Doesn't this mean they are both handled by the same IRQ? > > A side note of this is, multiple AHA-2940s would all be on the same IRQ also... > > Or is there some nuance of the PCI bus I am missing here.... yes there is, but not sure I can explain properly. my mptable: I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT conforms conforms 0 0 2 0 INT conforms conforms 0 1 2 1 INT conforms conforms 0 0 2 2 INT conforms conforms 0 3 2 3 INT conforms conforms 0 4 2 4 INT conforms conforms 0 5 2 5 INT conforms conforms 0 6 2 6 INT conforms conforms 0 7 2 7 INT conforms conforms 0 8 2 8 INT conforms conforms 0 9 2 9 INT conforms conforms 0 10 2 10 INT conforms conforms 0 11 2 11 INT conforms conforms 0 12 2 12 INT conforms conforms 0 13 2 13 INT conforms conforms 0 14 2 14 INT conforms conforms 0 15 2 15 >>> INT active-lo level 1 8:A 2 16 >>> INT active-lo level 1 9:A 2 17 >>> INT active-lo level 1 10:A 2 18 >>> INT active-lo level 1 12:A 2 19 SMI conforms conforms 0 0 2 23 note how the 4 PCI IRQs are all on INTA, but different PCI devices, and get directed to different APIC IRQs (16,17,18,19). my dmesg shows: de0 rev 17 int a irq 16 on pci0:8 ^ ^^ ^ ahc0 rev 0 int a irq 19 on pci0:12 ^ ^^ ^^ note that the de0 would have been redirected to ISA IRQ10 and the ahc0 would have been redirected to ISA IRQ11 on the standard kernel. The SMP kernel parses the mp table info and decides it can program the IO APIC to use these upper INTs (and eventually I'll reprogram the MD chipset to "undirect" the ISA INTs allowing other ISA cards to use them!) A few simple changes to pci.c and PCI INTs now appear on APIC IRQs 16-19. unfortunately not all MB manufacturers do it this way, often using the "redirected" ISA INTs, completely ignoring APIC pins 16-23.... for some more insight on just how different these motherboards can be check out the database of mp tables on: http://www.freebsd.org/~fsmp/SMP/mptable/mptable.html -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 16:06:35 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id QAA13741 for smp-outgoing; Thu, 5 Dec 1996 16:06:35 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id QAA13735 for ; Thu, 5 Dec 1996 16:06:31 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA15903; Thu, 5 Dec 1996 17:06:06 -0700 Message-Id: <199612060006.RAA15903@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 16:36:49 MST." <199612052336.QAA00497@fast.cs.utah.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 17:06:06 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > My main point is that you wouldn't be limited to 16 IRQ lines. I understand, I'm just pointing out that *1* IO APIC provides 24 IRQs, and that most MBs ignore most or all of the additional 8 that are available! -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 16:23:35 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id QAA14486 for smp-outgoing; Thu, 5 Dec 1996 16:23:35 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id QAA14480 for ; Thu, 5 Dec 1996 16:23:31 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA16023; Thu, 5 Dec 1996 17:21:32 -0700 Message-Id: <199612060021.RAA16023@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Thomas Pfenning cc: "'Chris Csanady'" , "'Peter Wemm'" , "'smp@freebsd.org'" Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 10:55:33 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 17:21:31 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > Doesn't the lock version in this mail actual trash the cached value for > bootlock on every spin? What about using MCS queueing locks to solve > both the cache trashing and the reentrance at the same time. ... > >>ENTRY(boot_lock) > >> /* This is the Intel reccomended semaphore method */ > >> movb $0xff, %al > >>2: > >> xchgb %al, bootlock /* xchg is implicitly locked */ > >> cmpb $0xff, %al > >> jz 2b > >> ret > >> > >>ENTRY(boot_unlock) > >> movb $0, %al > >> xchgb %al, bootlock /* xchg is implicitly locked */ > >> ret > >> > >> /* initial value is 0xff, or "busy" */ > >> .globl bootlock > >>bootlock: .byte 0xff no, %al has 0xff in it, the initial locked value is 0xff each spin puts %al (ie 0xff) into bootlock and bootlock into %al so bootlock remains 0xff until the unlock function is called the unlock functiom puts 0x00 into %al, then does the xchgb, putting the 0x00 into bootlock, and tossing the contents of %al as it returns. so the next spin on the lock by ONE AP puts %al (0xff everytime) into bootlock, and gets %al filled with the 0x00 placed there by unlock. the other APs get the 0xff placed there by the one successful xchgb that the 1st AP made. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 16:51:51 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id QAA15945 for smp-outgoing; Thu, 5 Dec 1996 16:51:51 -0800 (PST) Received: (from fsmp@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id QAA15938 for freebsd-smp; Thu, 5 Dec 1996 16:51:49 -0800 (PST) Date: Thu, 5 Dec 1996 16:51:49 -0800 (PST) From: Steve Passe Message-Id: <199612060051.QAA15938@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/kern kern_shutdown.c sys/i386/include smp.h sys/i386/i386 machdep.c mp_machdep.c mpboot.s Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk fsmp 96/12/05 16:51:49 Modified: kern kern_shutdown.c i386/include smp.h Log: moved "extern smp_active" to smp.h Revision Changes Path 1.7 +0 -3 sys/kern/kern_shutdown.c 1.25 +11 -7 sys/i386/include/smp.h Modified: i386/i386 machdep.c mp_machdep.c mpboot.s Log: fixed bug preventing >2 CPUs from running. added debug code for APs not properly starting to mpboot.s & mp_machdep.c. minor cleanup of machdep.c as reguards SMP. cleanup of mp_machdep.c Help from: Terje Normann Marthinussen "Erich Boleyn,,,," , smp@freebsd.org Revision Changes Path 1.33 +14 -13 sys/i386/i386/machdep.c 1.32 +419 -355 sys/i386/i386/mp_machdep.c 1.15 +34 -1 sys/i386/i386/mpboot.s From owner-freebsd-smp Thu Dec 5 16:58:31 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id QAA16304 for smp-outgoing; Thu, 5 Dec 1996 16:58:31 -0800 (PST) Received: from INET-04-IMC.itg.microsoft.com (mail4.microsoft.com [131.107.3.29]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id QAA16299 for ; Thu, 5 Dec 1996 16:58:29 -0800 (PST) Received: by INET-04-IMC.itg.microsoft.com with SMTP (Microsoft Exchange Server Internet Mail Connector Version 4.0.994.63) id <01BBE2CD.27DDF010@INET-04-IMC.itg.microsoft.com>; Thu, 5 Dec 1996 16:55:44 -0800 Message-ID: From: Thomas Pfenning To: "'Steve Passe'" Cc: "'Chris Csanady'" , "'Peter Wemm'" , "'smp@freebsd.org'" Subject: RE: make locking more generic? Date: Thu, 5 Dec 1996 16:49:30 -0800 X-Mailer: Microsoft Exchange Server Internet Mail Connector Version 4.0.994.63 Encoding: 68 TEXT Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hmm, but xchgb actually writes a new value into the cache line on each spin and dirties the cache line? So you are saying, the cache coherence protocol is smart enough to recognize it is the same value and does not actually require an update of the cache in a remote CPU spinning on the same lock. Or is it the xchgb instruction which does not write an 0xff if it is already there? The other thing I don't quite understand is the use of an atomic memory operation for the unlock. Cheers Thomas >-----Original Message----- >From: Steve Passe [SMTP:smp@csn.net] >Sent: Thursday, December 05, 1996 4:22 PM >To: Thomas Pfenning >Cc: 'Chris Csanady'; 'Peter Wemm'; 'smp@freebsd.org' >Subject: Re: make locking more generic? > >Hi, > >> Doesn't the lock version in this mail actual trash the cached value for >> bootlock on every spin? What about using MCS queueing locks to solve >> both the cache trashing and the reentrance at the same time. > ... >> >>ENTRY(boot_lock) >> >> /* This is the Intel reccomended semaphore method */ >> >> movb $0xff, %al >> >>2: >> >> xchgb %al, bootlock /* xchg is implicitly locked */ >> >> cmpb $0xff, %al >> >> jz 2b >> >> ret >> >> >> >>ENTRY(boot_unlock) >> >> movb $0, %al >> >> xchgb %al, bootlock /* xchg is implicitly locked */ >> >> ret >> >> >> >> /* initial value is 0xff, or "busy" */ >> >> .globl bootlock >> >>bootlock: .byte 0xff > >no, > > %al has 0xff in it, > the initial locked value is 0xff > each spin puts %al (ie 0xff) into bootlock and bootlock into %al > so bootlock remains 0xff until the unlock function is called > > the unlock functiom puts 0x00 into %al, then does the xchgb, putting > the 0x00 into bootlock, and tossing the contents of %al as it returns. > > so the next spin on the lock by ONE AP puts %al (0xff everytime) into > bootlock, and gets %al filled with the 0x00 placed there by unlock. > > the other APs get the 0xff placed there by the one successful xchgb > that the 1st AP made. > >-- >Steve Passe | powered by >smp@csn.net | FreeBSD > From owner-freebsd-smp Thu Dec 5 17:07:22 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id RAA16707 for smp-outgoing; Thu, 5 Dec 1996 17:07:22 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id RAA16702 for ; Thu, 5 Dec 1996 17:07:19 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id SAA16252; Thu, 5 Dec 1996 18:06:50 -0700 Message-Id: <199612060106.SAA16252@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Thomas Pfenning cc: "'Chris Csanady'" , "'Peter Wemm'" , "'smp@freebsd.org'" Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 16:49:30 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 18:06:50 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Hmm, > > but xchgb actually writes a new value into the cache line on each spin > and dirties the cache line? So you are saying, the cache coherence > protocol is smart enough to recognize it is the same value and does not > actually require an update of the cache in a remote CPU spinning on the > same lock. Or is it the xchgb instruction which does not write an 0xff > if it is already there? oh-oh, I didn't read close enough (alot of mail has backed up lately). forget what I said... cache, I couldn't say, it might have a bad effect but what can you do about it? a blocking lock as oppossed to a spin-lock? I think the theory is that the collissions are statistically so short that anything more complex would be a loosing situation. ---- > The other thing I don't quite understand is the use of an atomic memory > operation for the unlock. the atomic move may or may not be necessary. since its a byte there is no chance a misaligned lock being written in 2 bus operations. anyone know? -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Thu Dec 5 17:49:37 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id RAA18572 for smp-outgoing; Thu, 5 Dec 1996 17:49:37 -0800 (PST) Received: from INET-01-IMC.microsoft.com (mail1.microsoft.com [131.107.3.41]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id RAA18566 for ; Thu, 5 Dec 1996 17:49:35 -0800 (PST) Received: by INET-01-IMC.microsoft.com with SMTP (Microsoft Exchange Server Internet Mail Connector Version 4.0.994.63) id <01BBE2D4.9A72A790@INET-01-IMC.microsoft.com>; Thu, 5 Dec 1996 17:49:03 -0800 Message-ID: From: Thomas Pfenning To: "'Steve Passe'" Cc: "'Chris Csanady'" , "'Peter Wemm'" , "'smp@freebsd.org'" Subject: RE: make locking more generic? Date: Thu, 5 Dec 1996 17:49:23 -0800 X-Mailer: Microsoft Exchange Server Internet Mail Connector Version 4.0.994.63 Encoding: 59 TEXT Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Actually you can do something about it which does not cause any contention on the bus doing the update protocol and even grants the lock in fifo order. However, the lock actually transforms into a linked list and you spin on memory location in your local cache, so you want to make it aligned to cache lines. Mellor-Crummey and M. Scott, Algorithms for scalable synchronization on shared-memory multiprocessors; ACM Trans. Computer Systems, 9(1):21--65 (1991) Cheers Thomas >-----Original Message----- >From: Steve Passe [SMTP:smp@csn.net] >Sent: Thursday, December 05, 1996 5:07 PM >To: Thomas Pfenning >Cc: 'Chris Csanady'; 'Peter Wemm'; 'smp@freebsd.org' >Subject: Re: make locking more generic? > >> Hmm, >> >> but xchgb actually writes a new value into the cache line on each spin >> and dirties the cache line? So you are saying, the cache coherence >> protocol is smart enough to recognize it is the same value and does not >> actually require an update of the cache in a remote CPU spinning on the >> same lock. Or is it the xchgb instruction which does not write an 0xff >> if it is already there? > >oh-oh, I didn't read close enough (alot of mail has backed up lately). >forget what I said... > >cache, I couldn't say, it might have a bad effect but what can you do >about it? a blocking lock as oppossed to a spin-lock? I think the theory is >that the collissions are statistically so short that anything more complex >would be a loosing situation. > >---- >> The other thing I don't quite understand is the use of an atomic memory >> operation for the unlock. > >the atomic move may or may not be necessary. since its a byte there is >no chance a misaligned lock being written in 2 bus operations. anyone know? > >-- >Steve Passe | powered by >smp@csn.net | FreeBSD > >-----BEGIN PGP PUBLIC KEY BLOCK----- >Version: 2.6.2 > >mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE >04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX >WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR >tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ >=ds99 >-----END PGP PUBLIC KEY BLOCK----- > From owner-freebsd-smp Thu Dec 5 17:53:56 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id RAA18675 for smp-outgoing; Thu, 5 Dec 1996 17:53:56 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id RAA18670 for ; Thu, 5 Dec 1996 17:53:52 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id VAA23427; Thu, 5 Dec 1996 21:52:00 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612060152.VAA23427@bluenose.na.tuns.ca> Subject: Re: Good News & Bad News To: Tor.Egge@idt.ntnu.no (Tor Egge) Date: Thu, 5 Dec 1996 21:52:00 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612052230.XAA03136@pat.idt.unit.no> from Tor Egge at "Dec 5, 96 11:30:34 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > >the system reboots itself unpredictably shortly after the second > > > >CPU is activated. > > > > > > Just a thought, but how many of the machines besides the Tyan Titan > > > Pro have the BSP at other than CPU #0? The Titan Pro's BSD is > > > (physically) Processor #1. This is just the most obvious difference > > > between the Titan Pro and the other boards. > > > > from the mptab;e database: > > > > Asus P/I-P65UP5 > > Supermicro P6DNE > > > > does anyone running these machines have experience with the recent SMP kernel? > > I tried a few hours old kernel on an ASUS P/I-P65UP5, with APIC_IO > enabled. When compiling a new kernel with two active CPUs, I got > error messages from gcc, and the compile failed. Restarting the > $2 = 0xfff7eff4 > > This indicates an attempt to dereference address 0xfff7eff4, which seems > bogus :-( > > - Tor Egge > I turn on DDB, here is the report from DDB when system reboots shortly after the second CPU is launched: cpunumber =0 fault virtual address = 0xfffbeff4 fault code = supervisor read, page not present instruction pointer = 0x8:0xf01be11b stack pointer = 0x8:0xefbffe94 frame pointer = 0x10:0xefbffeb0 code segment = base 0x0, limit 0xffff, type0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL=0 current process = 270 (sh) interrupt mask = kernel: type 12 trap, code=0 stopped at _pmap_enter+0x8f: movl 0 (%ecx), %ecx db> trace _pmap_enter (f0cbc464,efbfd000,da5000,7,0) at _pmap_enter+0x8ff vm_fault(f0cbc400,efbfd000,3,0,0) at _vm_fault+0xd0b trap_pfault(efbfffbc,1) at _trap_pfault+0xd4 _trap(27,27,56000,509ec,efbfdb34) at _trap+0x146 calltrap() at calltrap+0x1a - trap 12, eip=0xb0f0, ebp=0xefbfdb34 --- - curporc=0xf0be3c00, pie=270 --- Jim From owner-freebsd-smp Thu Dec 5 17:56:57 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id RAA18875 for smp-outgoing; Thu, 5 Dec 1996 17:56:57 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id RAA18869 for ; Thu, 5 Dec 1996 17:56:51 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id JAA11590; Fri, 6 Dec 1996 09:55:12 +0800 (WST) Message-Id: <199612060155.JAA11590@spinner.DIALix.COM> To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: ccsanady@friley216.res.iastate.edu, smp@csn.net, smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 10:45:07 MST." <199612051745.KAA18517@fast.cs.utah.edu> Date: Fri, 06 Dec 1996 09:55:12 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Kevin Van Maren wrote: > Yes, the reason you need finer grained locking is because the > interrupts *should* go to the other processor. If one > processor is handling an interrupt and annother int comes > in, the other CPU should be able to handle it. This > would finally give parallel I/O! Linux doesn't do this, > and they do very poorly when not every process is CPU bound. > > Kevin > > ps: This will most likely mean fixing device drivers as well. Yes, it will most likely one of two options for each driver.. We will have to modify it to do fine grain locking (this is a major problem for the network cards due to the mbuf design), or have some way of running the driver in "backwards compatability" mode. Needless to say, we need to get more fundamental things like floating point working again first before we even consider this level of change. Cheers, -Peter From owner-freebsd-smp Thu Dec 5 18:05:53 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id SAA19271 for smp-outgoing; Thu, 5 Dec 1996 18:05:53 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id SAA19266 for ; Thu, 5 Dec 1996 18:05:47 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id KAA11744; Fri, 6 Dec 1996 10:04:38 +0800 (WST) Message-Id: <199612060204.KAA11744@spinner.DIALix.COM> To: Terry Lambert cc: smp@csn.net (Steve Passe), ccsanady@friley216.res.iastate.edu, smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 11:37:58 MST." <199612051837.LAA19621@phaeton.artisoft.com> Date: Fri, 06 Dec 1996 10:04:38 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Terry Lambert wrote: > > So whenever a CPU acquires the lock I have > > the code set the priority of that CPU to "VERY LOW". So now all pending I NTs > > will be sent to it, which it can deal with since it already holds the lock. > > When a CPU releaes the lock it resets its priority to "NORMAL". > > > > The problem is that whenever a CPU gets an INT its PPR is increased to > > the priority of the INT, thus negating the "VERY LOW" prio I set in the > > TPR. So if a CPU is behind the lock because of an INT it WON'T get > > sent other pending INTs, the other CPU will. Now that we have IPIs > > working I hope to be able to send an IPI to the other CPUs telling them to set > > their TPR to "VERY HIGH", thus dis-abling them from getting INTs. > > I don't understand. > > If CPU 1 is servicing INT 12 from a disk controller, and you get INT 4 > from a serial port, don't you *want* CPU 2 to service the INT 4 in > parallel??? Yes, it would be nice, **if** the second cpu can enter the kernel. Since it can't at present, it has to wait for the first cpu to leave. However if the cpu that was servicing the interrupt in the first place gets the irq4, it can process it immediately. > > Terry Lambert > terry@lambert.org Cheers, -Peter From owner-freebsd-smp Thu Dec 5 18:19:10 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id SAA19818 for smp-outgoing; Thu, 5 Dec 1996 18:19:10 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id SAA19806 for ; Thu, 5 Dec 1996 18:19:06 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vVpwO-00066Q-00; Thu, 5 Dec 1996 18:23:04 -0800 To: smp@csn.net cc: smp@freebsd.org Subject: All 4 CPUs online! Date: Thu, 05 Dec 1996 18:23:03 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I CVSUPed the changes made today, and my box comes up with all 4 CPUs now (after executing "sysctl -w kern.smp_active=4"). The end of my dmesg log looks like: stray irq 13 changing root device to sd0a Enabled INTs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13, 14, 15, imen: 0x00ff1c01 SMP: All idle procs online. SMP: Starting 1st AP! SMP: AP CPU #1 LAUNCHED!! Starting Scheduling... SMP: TADA! CPU #1 made it into the scheduler!. SMP: 2 of 4 CPU's online. Unlocking next CPU.. SMP: AP CPU #2 LAUNCHED!! Starting Scheduling... SMP: TADA! CPU #2 made it into the scheduler!. SMP: 3 of 4 CPU's online. Unlocking next CPU.. SMP: AP CPU #3 LAUNCHED!! Starting Scheduling... SMP: TADA! CPU #3 made it into the scheduler!. SMP: All 4 CPU's are online! Great job Steve! I'll beat on it with parallel compiles... -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Thu Dec 5 18:30:32 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id SAA20339 for smp-outgoing; Thu, 5 Dec 1996 18:30:32 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id SAA20334 for ; Thu, 5 Dec 1996 18:30:29 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id UAA00386 for ; Thu, 5 Dec 1996 20:30:21 -0600 (CST) Message-Id: <199612060230.UAA00386@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: freebsd-smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of Fri, 06 Dec 1996 09:55:12 +0800. <199612060155.JAA11590@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 20:30:20 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >Kevin Van Maren wrote: >> Yes, the reason you need finer grained locking is because the >> interrupts *should* go to the other processor. If one >> processor is handling an interrupt and annother int comes >> in, the other CPU should be able to handle it. This >> would finally give parallel I/O! Linux doesn't do this, >> and they do very poorly when not every process is CPU bound. >> >> Kevin >> >> ps: This will most likely mean fixing device drivers as well. > >Yes, it will most likely one of two options for each driver.. We will >have to modify it to do fine grain locking (this is a major problem for >the network cards due to the mbuf design), or have some way of running s/design/stupidity/ Chris >the driver in "backwards compatability" mode. > >Needless to say, we need to get more fundamental things like floating point >working again first before we even consider this level of change. > >Cheers, >-Peter From owner-freebsd-smp Thu Dec 5 19:03:42 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id TAA21925 for smp-outgoing; Thu, 5 Dec 1996 19:03:42 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id TAA21860 for ; Thu, 5 Dec 1996 19:03:31 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id LAA12276; Fri, 6 Dec 1996 11:02:55 +0800 (WST) Message-Id: <199612060302.LAA12276@spinner.DIALix.COM> To: Chris Csanady cc: freebsd-smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of "Thu, 05 Dec 1996 20:30:20 CST." <199612060230.UAA00386@friley216.res.iastate.edu> Date: Fri, 06 Dec 1996 11:02:54 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Chris Csanady wrote: > >Kevin Van Maren wrote: > >> Yes, the reason you need finer grained locking is because the > >> interrupts *should* go to the other processor. If one > >> processor is handling an interrupt and annother int comes > >> in, the other CPU should be able to handle it. This > >> would finally give parallel I/O! Linux doesn't do this, > >> and they do very poorly when not every process is CPU bound. > >> > >> Kevin > >> > >> ps: This will most likely mean fixing device drivers as well. > > > >Yes, it will most likely one of two options for each driver.. We will > >have to modify it to do fine grain locking (this is a major problem for > >the network cards due to the mbuf design), or have some way of running > > s/design/stupidity/ > > Chris Well, there are several things to consider before writing it off as "stupid", it's bloody quick compared to what something like STREAMS would be like, which is much more MP-safe. With the present "design", an mbuf is allocated in the network soft interrupt layer, and that mbuf is passed all the way up to the socket buffers, right through the ip and tcp layers without queueing it anywhere. The netisr handler literally "runs" the upwards direction tcp/ip "engine". It's pretty much the same in the other direction, but from memory one direction has one queueing stage, the other doesn't. STREAMS, on the other hand, has "modules" with an incoming and outgoing queue. The network driver allocates an mblk/dblk and queues it in it's upstream neighbor's queue. The upstream module's service routine is run which dequeues it, processes it, and enqueues it on the upstream side... And so on right up to the stream head. On the plus side, it's got a lot of potential for putting hooks in for fine-grain locking on each queue and data block, but the overheads are incredible. I'm not familiar with Linux's design, I think that it's a simplified (read: the crap has been stripped out and optimised) version of the mbuf cluster model, where large buffers are used and passed around. I do not know if they queue on entry to each logical "component" of the protocol stack, but I suspect not, since they are after speed. Calling it a simplified version of mbuf clusters is probably not going to be popular, but that's what a casual glance suggested to me. We have two major types of mbufs.. "standard" small ones 128 bytes long, with about 106 bytes of data space or so, and "clusters" where the mbuf points to a 2K or 4K page of data. I think the Linux model is like the latter where there is either a seperate header and data chunk, or the header is at the start of the data "page". If this is the case, their system probably won't lend itself to multi-threading any better than ours. I suspect that we're going to be stuck with a giant "networking lock" to cover everything from the soft network interrupts through the mbuf code, through to the protocol engine and high level socket buffers. There may be room to have a decoupling layer in between the network cards and the network "protocol engine" as such, and the same at the top end. This may allow us to get away with running the soft net processing for the cards in parallel with the network "engine" as such. This will require a locked queueing stage to get data from one to the other. Cheers, -Peter From owner-freebsd-smp Thu Dec 5 19:07:08 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id TAA22837 for smp-outgoing; Thu, 5 Dec 1996 19:07:08 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id TAA22803 for ; Thu, 5 Dec 1996 19:07:03 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id UAA16913; Thu, 5 Dec 1996 20:06:51 -0700 Message-Id: <199612060306.UAA16913@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@FreeBSD.ORG cc: Tor.Egge@idt.ntnu.no (Tor Egge), "J.M. Chuang" , Janick.Taillandier@ratp.fr (Janick TAILLANDIER), Peter Wemm Subject: last major problem In-reply-to: Your message of "Thu, 05 Dec 1996 21:52:00 -0400." <199612060152.VAA23427@bluenose.na.tuns.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 20:06:51 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, It appears we have one last serious problem to fix. 3 users report crashing shortly after starting the 2nd CPU. -- >I tried a few hours old kernel on an ASUS P/I-P65UP5, with APIC_IO >enabled. When compiling a new kernel with two active CPUs, I got >error messages from gcc, and the compile failed. Restarting the >kernel compiling caused a trap 12, and a kernel dump. > >When looking at the kernel dump, I get > >#0 boot (howto=256) at ../../kern/kern_shutdown.c:267 >#1 0xe0112d29 in panic (fmt=0xe01bcbcf "page fault") > at ../../kern/kern_shutdown.c:395 >#2 0xe01bd8b5 in trap_fatal (frame=0xdfbffe58) at ../../i386/i386/trap.c:747 >#3 0xe01bd2e8 in trap_pfault (frame=0xdfbffe58, usermode=0) > at ../../i386/i386/trap.c:654 >#4 0xe01bcf1b in trap (frame={tf_es = -270335984, tf_ds = 16, > tf_edi = -270395292, tf_esi = -541077504, tf_ebp = -541065552, > tf_isp = -541065600, tf_ebx = 296243200, tf_edx = -4194304, > tf_ecx = -528396, tf_eax = -528396, tf_trapno = 12, tf_err = 0, > tf_eip = -535058833, tf_cs = 8, tf_eflags = 66178, tf_esp = -530083069, > tf_ss = -270296448}) at ../../i386/i386/trap.c:313 >#5 0xe01ba66f in pmap_enter (pmap=0xefe21864, va=3753889792, pa=296243200, > prot=7 '\a', wired=0) at ../../i386/i386/pmap.c:2014 >#6 0xe01a4153 in vm_fault (map=0xefe21800, vaddr=3753889792, > fault_type=3 '\003', change_wiring=0) at ../../vm/vm_fault.c:773 >#7 0xe01bd240 in trap_pfault (frame=0xdfbfffbc, usermode=1) > at ../../i386/i386/trap.c:634 >#8 0xe01bcdc3 in trap (frame={tf_es = 39, tf_ds = 39, tf_edi = 352256, > tf_esi = 330220, tf_ebp = -541074428, tf_isp = -541065244, tf_ebx = 0, > tf_edx = 1, tf_ecx = 330220, tf_eax = 0, tf_trapno = 12, tf_err = 7, > tf_eip = 45296, tf_cs = 31, tf_eflags = 66050, tf_esp = -541074452, > tf_ss = 39}) at ../../i386/i386/trap.c:241 >(kgdb) up 5 >#5 0xe01ba66f in pmap_enter (pmap=0xefe21864, va=3753889792, pa=296243200, > prot=7 '\a', wired=0) at ../../i386/i386/pmap.c:2014 >2014 origpte = *(vm_offset_t *)pte; >(kgdb) print/x pte >$2 = 0xfff7eff4 > >This indicates an attempt to dereference address 0xfff7eff4, which seems >bogus :-( > >- Tor Egge --- > I turn on DDB, here is the report from DDB when system reboots shortly after > the second CPU is launched: > > cpunumber =0 > fault virtual address = 0xfffbeff4 > fault code = supervisor read, page not present > instruction pointer = 0x8:0xf01be11b > stack pointer = 0x8:0xefbffe94 > frame pointer = 0x10:0xefbffeb0 > code segment = base 0x0, limit 0xffff, type0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, resume, IOPL=0 > current process = 270 (sh) > interrupt mask = > kernel: type 12 trap, code=0 > stopped at _pmap_enter+0x8f: movl 0 (%ecx), %ecx > db> trace > _pmap_enter (f0cbc464,efbfd000,da5000,7,0) at _pmap_enter+0x8ff > vm_fault(f0cbc400,efbfd000,3,0,0) at _vm_fault+0xd0b > trap_pfault(efbfffbc,1) at _trap_pfault+0xd4 > _trap(27,27,56000,509ec,efbfdb34) at _trap+0x146 > calltrap() at calltrap+0x1a > - trap 12, eip=0xb0f0, ebp=0xefbfdb34 --- > - curporc=0xf0be3c00, pie=270 --- > > > Jim --- >I have installed the latest patches and I am still having trap 12 >a few seconds after starting the second CPU. Everything seems ok >with only one CPU. >The machine is a Dell Optiplex GXpro with 2 P6 200 MHz, 32MB, >SCSI disk on Adaptec 2940UW. > ... >Stopped at _pmap_enter+0x8f: movl 0(%ecx),%ecx > >trace gives: > >_pmap_enter(f13ad064,efbfd000,17c8000,7,0) at _pmap_enter+0x8f >_vm_fault(f13ad000,efbfd000,3,0,0) at _vm_fault+xxd`b >_trap_pfault(efbfffbc,1) at _trap_pfault+0xd4 >_trap(27,27,50274,1,efbfd5F4) at _trap+0x14b >calltrap() at calltrap+0x1a >--- trap 12, eip = 0xe9fc, ebp =0xefbfd5f4 --- >--- curproc = 0xf13e9a00, pid = 1561 --- > >I have similar crash without APIC_IO or SMP_INVLTLB. > >Janick --- Tor's machine: P6, Asus P/I-P65UP5/C-P6ND, entry 5 in mptable database Jims's machine: P6, Titan Pro, entry 17 in mptable database (someone else, same MB) (I'm assumming its the Titan Pro, Tomcat II/EIDE previously reported working) Janick's machine: P6, Dell Optiplex GXPro, entry 6 in mptable database. They're all P6, ... any theories/clues??? -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 19:18:49 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id TAA24555 for smp-outgoing; Thu, 5 Dec 1996 19:18:49 -0800 (PST) Received: from pat.idt.unit.no (0@pat.idt.unit.no [129.241.103.5]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id TAA24539 for ; Thu, 5 Dec 1996 19:18:42 -0800 (PST) Received: from idt.unit.no (tegge@ikke.idt.unit.no [129.241.111.65]) by pat.idt.unit.no (8.8.4/8.8.4) with ESMTP id EAA09185; Fri, 6 Dec 1996 04:18:29 +0100 (MET) Message-Id: <199612060318.EAA09185@pat.idt.unit.no> To: smp@csn.net Cc: smp@FreeBSD.ORG, smp@bluenose.na.tuns.ca, Janick.Taillandier@ratp.fr, peter@spinner.dialix.com Subject: Re: last major problem In-Reply-To: Your message of "Thu, 05 Dec 1996 20:06:51 -0700" References: <199612060306.UAA16913@clem.systemsix.com> X-Mailer: Mew version 1.06 on Emacs 19.33.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 06 Dec 1996 04:18:29 +0100 From: Tor Egge Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > Hi, > > It appears we have one last serious problem to fix. 3 users report > crashing shortly after starting the 2nd CPU. Two data points: 1. I did not have options SMP_INVLTLB in my kernel config file. 2. I'll report back when I've tried the following patch: Index: pmap.c =================================================================== RCS file: /export/akg1/smp-cvs/sys/i386/i386/pmap.c,v retrieving revision 1.31 diff -c -r1.31 pmap.c *** pmap.c 1996/12/03 05:51:12 1.31 --- pmap.c 1996/12/06 03:09:33 *************** *** 552,557 **** --- 552,560 ---- if (frame != (((unsigned) APTDpde) & PG_FRAME)) { APTDpde = (pd_entry_t) (frame | PG_RW | PG_V); invltlb(); + #if defined(SMP_INVLTLB) + smp_invltlb(); + #endif } return (unsigned *) APTmap; } - Tor Egge From owner-freebsd-smp Thu Dec 5 19:23:54 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id TAA24890 for smp-outgoing; Thu, 5 Dec 1996 19:23:54 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id TAA24885 for ; Thu, 5 Dec 1996 19:23:49 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id UAA17068; Thu, 5 Dec 1996 20:23:41 -0700 Message-Id: <199612060323.UAA17068@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Erich Boleyn cc: smp@freebsd.org Subject: Re: All 4 CPUs online! In-reply-to: Your message of "Thu, 05 Dec 1996 18:23:03 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 20:23:40 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > > I CVSUPed the changes made today, and my box comes up with all 4 CPUs > now (after executing "sysctl -w kern.smp_active=4"). > > The end of my dmesg log looks like: > ... > SMP: All 4 CPU's are online! > > Great job Steve! credit for most of what makes >2 CPUs work goes to Peter, I just found *MY* bug that was keeping it from happening. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 19:36:06 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id TAA25621 for smp-outgoing; Thu, 5 Dec 1996 19:36:06 -0800 (PST) Received: from friley216.res.iastate.edu (friley216.res.iastate.edu [129.186.78.216]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id TAA25594 for ; Thu, 5 Dec 1996 19:35:59 -0800 (PST) Received: from friley216.res.iastate.edu (loopback [127.0.0.1]) by friley216.res.iastate.edu (8.8.3/8.7.3) with ESMTP id VAA00697; Thu, 5 Dec 1996 21:35:29 -0600 (CST) Message-Id: <199612060335.VAA00697@friley216.res.iastate.edu> X-Mailer: exmh version 1.6.9 8/22/96 To: Peter Wemm cc: freebsd-smp@freebsd.org Subject: Re: make locking more generic? In-reply-to: Your message of Fri, 06 Dec 1996 11:02:54 +0800. <199612060302.LAA12276@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 21:35:29 -0600 From: Chris Csanady Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >Well, there are several things to consider before writing it off as >"stupid", it's bloody quick compared to what something like STREAMS >would be like, which is much more MP-safe. I would not say so in comparison with that. :) Just that it could be done better.. >With the present "design", an mbuf is allocated in the network soft >interrupt layer, and that mbuf is passed all the way up to the socket >buffers, right through the ip and tcp layers without queueing it anywhere. >The netisr handler literally "runs" the upwards direction tcp/ip "engine". >It's pretty much the same in the other direction, but from memory one >direction has one queueing stage, the other doesn't. > >STREAMS, on the other hand, has "modules" with an incoming and outgoing >queue. The network driver allocates an mblk/dblk and queues it in it's >upstream neighbor's queue. The upstream module's service routine is run >which dequeues it, processes it, and enqueues it on the upstream side... >And so on right up to the stream head. On the plus side, it's got a lot >of potential for putting hooks in for fine-grain locking on each queue and >data block, but the overheads are incredible. > >I'm not familiar with Linux's design, I think that it's a simplified >(read: the crap has been stripped out and optimised) version of the >mbuf cluster model, where large buffers are used and passed around. >I do not know if they queue on entry to each logical "component" of >the protocol stack, but I suspect not, since they are after speed. > >Calling it a simplified version of mbuf clusters is probably not going >to be popular, but that's what a casual glance suggested to me. We have >two major types of mbufs.. "standard" small ones 128 bytes long, with about >106 bytes of data space or so, and "clusters" where the mbuf points to a >2K or 4K page of data. I think the Linux model is like the latter where >there is either a seperate header and data chunk, or the header is at the >start of the data "page". If this is the case, their system probably won't >lend itself to multi-threading any better than ours. > >I suspect that we're going to be stuck with a giant "networking lock" >to cover everything from the soft network interrupts through the mbuf >code, through to the protocol engine and high level socket buffers. Perhaps this would be fixed as a side effect to implementing some of the things that Van Jacobson was talking about. I believe the structure that he talks about would lend itself fairly well to finer grained locking. (I could be wrong..) But anyway, for those who are interested.. http://www.noao.edu/~rstevens/vanj.93sep07.txt ftp://ftp.ee.lbl.gov/talks/vj-nkarch.ps.Z ftp://ftp.ee.lbl.gov/talks/vj-nws93-1.ps.Z This sounds like it would be a fun project. Although, currently I dont know much about our net code. I hope to read TCP/IP Illustrated 1 & 2 over christmas break though. :) Chris > >There may be room to have a decoupling layer in between the network cards >and the network "protocol engine" as such, and the same at the top end. >This may allow us to get away with running the soft net processing for the >cards in parallel with the network "engine" as such. This will require >a locked queueing stage to get data from one to the other. > >Cheers, >-Peter From owner-freebsd-smp Thu Dec 5 21:18:42 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id VAA28630 for smp-outgoing; Thu, 5 Dec 1996 21:18:42 -0800 (PST) Received: from pat.idt.unit.no (0@pat.idt.unit.no [129.241.103.5]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id VAA28625 for ; Thu, 5 Dec 1996 21:18:39 -0800 (PST) Received: from idt.unit.no (tegge@presis.idt.ntnu.no [129.241.111.173]) by pat.idt.unit.no (8.8.4/8.8.4) with ESMTP id GAA10854; Fri, 6 Dec 1996 06:18:30 +0100 (MET) Message-Id: <199612060518.GAA10854@pat.idt.unit.no> To: smp@csn.net Cc: smp@FreeBSD.ORG, smp@bluenose.na.tuns.ca, Janick.Taillandier@ratp.fr, peter@spinner.dialix.com Subject: Re: last major problem In-Reply-To: Your message of "Fri, 06 Dec 1996 04:18:29 +0100" X-Mailer: Mew version 1.06 on Emacs 19.33.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 06 Dec 1996 06:18:29 +0100 From: Tor Egge Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk I wrote: > 2. I'll report back when I've tried the following patch: [ patch deleted ] I still get the same crash, with APIC_IO and SMP_INVLTLB defined :-(. - Tor Egge From owner-freebsd-smp Thu Dec 5 21:24:15 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id VAA28846 for smp-outgoing; Thu, 5 Dec 1996 21:24:15 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id VAA28841 for ; Thu, 5 Dec 1996 21:24:11 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id BAA24385; Fri, 6 Dec 1996 01:21:54 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612060521.BAA24385@bluenose.na.tuns.ca> Subject: Re: last major problem To: Tor.Egge@idt.ntnu.no (Tor Egge) Date: Fri, 6 Dec 1996 01:21:53 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612060518.GAA10854@pat.idt.unit.no> from Tor Egge at "Dec 6, 96 06:18:29 am" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > I wrote: > > 2. I'll report back when I've tried the following patch: > > [ patch deleted ] > > I still get the same crash, with APIC_IO and SMP_INVLTLB defined :-(. > I tried your patch, no luck for Titan pro as well! BTW, I found that in the `make depend' (mkdep) for compiling a kernel, the system crashes. Jim From owner-freebsd-smp Thu Dec 5 22:27:02 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id WAA00440 for smp-outgoing; Thu, 5 Dec 1996 22:27:02 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id WAA00435 for ; Thu, 5 Dec 1996 22:26:57 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id XAA17874 for ; Thu, 5 Dec 1996 23:26:54 -0700 Message-Id: <199612060626.XAA17874@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@FreeBSD.ORG Subject: Re: last major problem In-reply-to: Your message of "Thu, 05 Dec 1996 20:06:51 MST." <199612060306.UAA16913@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 23:26:54 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, so all 3 failing systems are P6. is there anyone now successfully running APIC_IO on a P6 system? -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Dec 5 22:41:49 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id WAA00797 for smp-outgoing; Thu, 5 Dec 1996 22:41:49 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id WAA00788 for ; Thu, 5 Dec 1996 22:41:41 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id OAA13185; Fri, 6 Dec 1996 14:41:13 +0800 (WST) Message-Id: <199612060641.OAA13185@spinner.DIALix.COM> To: "J.M. Chuang" cc: Tor.Egge@idt.ntnu.no (Tor Egge), smp@freebsd.org Subject: Re: last major problem In-reply-to: Your message of "Fri, 06 Dec 1996 01:21:53 -0400." <199612060521.BAA24385@bluenose.na.tuns.ca> Date: Fri, 06 Dec 1996 14:41:12 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk "J.M. Chuang" wrote: > > > > I wrote: > > > 2. I'll report back when I've tried the following patch: > > > > [ patch deleted ] > > > > I still get the same crash, with APIC_IO and SMP_INVLTLB defined :-(. > > > I tried your patch, no luck for Titan pro as well! > > BTW, I found that in the `make depend' (mkdep) for compiling a kernel, > the system crashes. > > Jim BTW, we discovered that there are some differences between the P5 and P6 class cpu's in the prefetch behavior that have bitten us before. If anybody has ever wondered what pmap_bootstrap2() was for, this is it. Under the P5 cpu's, the prefetch queue was filled, but under the P6, the prefetch queue or pipeline or whatever seems to be flushed on a load of %cr3. We were doing this sort of thing while running in low memory at 1MB: movl $_IdlePTD,%eax movl %eax,%cr3 pushl $MPbegin ret MPbegin: .. rest of boot code... The problem was that %eip was low, and loading %cr3 caused it to be running from unmapped memory. On the P5, there were enough instructions in the pipeline/prefetch queue to push the address and jump up to the high memory address without faulting. On the P6, the next instruction after the load of %cr3 faults immediately. I wonder if we're running into something like this? Things seem to be working with the P5 cpu's... I must look over the crash reports again to see if I can spot the reason why... Cheers, -Peter From owner-freebsd-smp Thu Dec 5 22:43:44 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id WAA00868 for smp-outgoing; Thu, 5 Dec 1996 22:43:44 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id WAA00860 for ; Thu, 5 Dec 1996 22:43:37 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id OAA13234; Fri, 6 Dec 1996 14:43:24 +0800 (WST) Message-Id: <199612060643.OAA13234@spinner.DIALix.COM> To: Steve Passe cc: smp@FreeBSD.ORG Subject: Re: last major problem In-reply-to: Your message of "Thu, 05 Dec 1996 23:26:54 MST." <199612060626.XAA17874@clem.systemsix.com> Date: Fri, 06 Dec 1996 14:43:24 +0800 From: Peter Wemm Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Steve Passe wrote: > Hi, > > so all 3 failing systems are P6. is there anyone now successfully running > APIC_IO on a P6 system? Wait a sec, I thought this was a problem caused by the SMP_INVLTLB code? Or is it a generic "P6 dislikes APIC_IO in general" problem? Cheers, -Peter From owner-freebsd-smp Thu Dec 5 22:47:50 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id WAA00962 for smp-outgoing; Thu, 5 Dec 1996 22:47:50 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id WAA00957 for ; Thu, 5 Dec 1996 22:47:46 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id XAA18003; Thu, 5 Dec 1996 23:47:35 -0700 Message-Id: <199612060647.XAA18003@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Peter Wemm cc: smp@FreeBSD.ORG Subject: Re: last major problem In-reply-to: Your message of "Fri, 06 Dec 1996 14:43:24 +0800." <199612060643.OAA13234@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 05 Dec 1996 23:47:35 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > Steve Passe wrote: > > Hi, > > > > so all 3 failing systems are P6. is there anyone now successfully running > > APIC_IO on a P6 system? > > Wait a sec, I thought this was a problem caused by the SMP_INVLTLB code? > Or is it a generic "P6 dislikes APIC_IO in general" problem? thats what I'm wondering, I've been so busy fighting the other fires that I haven't kept up on the details of this and may misunderstand... so additionally, has ANYONE EVER run a 'solid' APIC_IO kernel on a P6? -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Fri Dec 6 03:18:05 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id DAA04784 for smp-outgoing; Fri, 6 Dec 1996 03:18:05 -0800 (PST) Received: from bluenose.na.tuns.ca (bluenose.na.tuns.ca [134.190.50.156]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id DAA04778 for ; Fri, 6 Dec 1996 03:18:02 -0800 (PST) Received: (from smp@localhost) by bluenose.na.tuns.ca (8.7.6/8.7.3) id HAA25054; Fri, 6 Dec 1996 07:16:20 -0400 (AST) From: "J.M. Chuang" Message-Id: <199612061116.HAA25054@bluenose.na.tuns.ca> Subject: Re: last major problem To: peter@spinner.dialix.com (Peter Wemm) Date: Fri, 6 Dec 1996 07:16:19 -0400 (AST) Cc: smp@freebsd.org In-Reply-To: <199612060641.OAA13185@spinner.DIALix.COM> from Peter Wemm at "Dec 6, 96 02:41:12 pm" X-Mailer: ELM [version 2.4ME+ PL13 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > I still get the same crash, with APIC_IO and SMP_INVLTLB defined :-(. > > > > > I tried your patch, no luck for Titan pro as well! > > > > BTW, I found that in the `make depend' (mkdep) for compiling a kernel, > > the system crashes. > > > > Jim > > BTW, we discovered that there are some differences between the P5 and P6 > class cpu's in the prefetch behavior that have bitten us before. > > If anybody has ever wondered what pmap_bootstrap2() was for, this is it. > > Under the P5 cpu's, the prefetch queue was filled, but under the P6, the > prefetch queue or pipeline or whatever seems to be flushed on a load > of %cr3. > > We were doing this sort of thing while running in low memory at 1MB: > movl $_IdlePTD,%eax > movl %eax,%cr3 > pushl $MPbegin > ret > MPbegin: > .. rest of boot code... > > The problem was that %eip was low, and loading %cr3 caused it to be > running from unmapped memory. On the P5, there were enough instructions > in the pipeline/prefetch queue to push the address and jump up to the > high memory address without faulting. On the P6, the next instruction > after the load of %cr3 faults immediately. > > I wonder if we're running into something like this? Things seem to be > working with the P5 cpu's... I must look over the crash reports again to > see if I can spot the reason why... For compiling a kernel, `make depend' crashes the system. If 'make depend' is executed before `sysctl -w kern.smp_active=2', the kernel can be compiled to the very end until ------------- cc -c -x assembler-with-cpp -DLOCORE -nostdinc -I- -I. -I../.. -I../../../include -DFAILSAFE -DCOMPAT_43 -DCD9660 -DMSDOSFS -DNFS -DFFS -DINET -DKERNEL ../../i386/i386/swtch.s --- vers.o --- sh ../../conf/newvers.sh SMP -DFAILSAFE -DCOMPAT_43 -DCD9660 -DMSDOSFS -DNFS -DFFS -DINET :-( ------------- I can provide the ktrace file, if you need it to identify the problem. Jim Jim From owner-freebsd-smp Fri Dec 6 09:37:23 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA01932 for smp-outgoing; Fri, 6 Dec 1996 09:37:23 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id JAA01925 for ; Fri, 6 Dec 1996 09:37:13 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id BAA18860; Sat, 7 Dec 1996 01:36:44 +0800 (WST) Message-Id: <199612061736.BAA18860@spinner.DIALix.COM> To: Erich Boleyn cc: Steve Passe , smp@freebsd.org Subject: Re: P6 and FreeBSD/SMP (was -> Re: last major problem) In-reply-to: Your message of "Fri, 06 Dec 1996 08:30:17 PST." Date: Sat, 07 Dec 1996 01:36:44 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Erich Boleyn wrote: > > Steve Passe writes: > > > > so all 3 failing systems are P6. is there anyone now successfully runn ing > > > > APIC_IO on a P6 system? > > > > > > Wait a sec, I thought this was a problem caused by the SMP_INVLTLB code? > > > Or is it a generic "P6 dislikes APIC_IO in general" problem? > > > > thats what I'm wondering, I've been so busy fighting the other fires that > > I haven't kept up on the details of this and may misunderstand... > > > > so additionally, has ANYONE EVER run a 'solid' APIC_IO kernel on a P6? > > Hmmm... I think there are several issues afoot here. > > (1) Does the P6 run an APIC_IO kernel OK before activating the other CPUs > (2) Does the P6 run an APIC_IO kernel OK after activating the other CPUs > > ...and seen in another message... > > (3) [the thing about relying on the short prefect queue of the Pentium, and > the P6 might break it] > > I ran a bunch of tests last night and this morning with the kernel tree > from yesterday afternoon/evening. My results were (using APIC_IO + > SMP_INVLTLB) : > > -- After I activate the other 3 CPUs (via "sysctl -w kern.smp_active=4"), > standard (sinple) operations seem OK, but when I start compiling a > kernel, anywhere from 1/3 to 1/2 the way through it dies with a > "kernel trap 12: supervisor read/write, page not present" break to > the kernel debugger, going into pmap_enter. It is always that error > (I think I've seen a "write" once, with it saying a "read" trap the > other times). This implies that the answer to #2 is "no" (at least > on my test box). I tried this about a dozen times, with the same > results each time. Question: what are the offending faulting addresses? We use two PTD slots, one for accessing the "current" process (addresses 0xefc00000-0xeffeffff), and the other being for an "alternate" process or address space (range: 0xffc00000 - 0xfffeffff). These 4MB chunks are a sparse end-to-end set of page table pages representing the 4GB of process address space. In the one message that I have handy, something is very wrong. The "current" process has faulted on it's first stack page, and pmap_enter is somehow using the alternate APTD stuff for some reason. I do not understand how this comes about yet. > -- If I don't activate the other CPUs, I can do a dozen builds in a raw > with no problems. This implies the answwer to #1 is "yes". How does it go without SMP_INVLTLB BTW? Do you use a scsi or ide system? I think we've pretty much discovered that the IDE driver is very vulnerable to missing the invltlb calls on the alternate cpu's for some reason. > This leads me to believe that the problem is in the MP handling, not the > base APIC_IO stuff. Maybe this is a good time to tell some of us what > is different when activating the other CPUs as far as APIC control? I am suspicious about the APTD handling. I am wondering if we need to make the handling of the APTD pointer per-cpu or something. Then there's the CADDR1, CADDR2 etc stuff. There's plenty of potential targets to check out, but we need some more clues to work on first. > I've been digging through the source, and it seems that the "smp_invltlb" > is separate from the normal "invltlb" function? There are *many* places > where "invltlb_1pg" (I think that was it) or other variants are called and > no SMP invalidates are propagated. This strikes me as a situation > fraught with potential (and currently real) problems. What is the > design goal here? smp_invltlb() is called from the invltlb() function. When compiling with SMP_INVLTLB, invltlb() is no longer inlined.. it's a called function in mp_machdep.c Yes, this is extreme overkill, probably 90% of those calls to smp_invltlb() are unnecessary.. They should not be harmful, but once it's working we can optimise it quite a lot. > On a side note (issue #3 I saw a comment on), first of all, never, *ever* > rely on short prefetch queues. A proper sequence which flushes the queues > in the appropriate places should be used. The Intel manuals have many > examples of this. I'll take a look at the code sequence and see what's > up with it. It could be that the comment that was made about it was > bogus. (if not, then that could very well be a fatal problem. Certainly > the P6 and what I've heard of the next project will become more and more > unpredictable unless the proper methods to serialize control register > changes are used). urk, sorry if I gave the impression that we were deliberately doing this.. No, it was an *accident* that the early smp code did this, it worked fine on the P5, but failed on the P6. I was thinking out aloud to the effect of "Hmm, I wonder if there's somewhere else that this kind of thing is happening that we're not aware of?". > -- > Erich Stefan Boleyn \_ E-mail (preferred): > Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) > Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying " Cheers, -Peter From owner-freebsd-smp Fri Dec 6 09:58:34 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA02698 for smp-outgoing; Fri, 6 Dec 1996 09:58:34 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id JAA02692 for ; Fri, 6 Dec 1996 09:58:26 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA20991; Fri, 6 Dec 1996 10:57:26 -0700 Message-Id: <199612061757.KAA20991@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Erich Boleyn cc: Peter Wemm , smp@freebsd.org Subject: Re: P6 and FreeBSD/SMP (was -> Re: last major problem) In-reply-to: Your message of "Fri, 06 Dec 1996 08:30:17 PST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 06 Dec 1996 10:57:25 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >This leads me to believe that the problem is in the MP handling, not the >base APIC_IO stuff. Maybe this is a good time to tell some of us what >is different when activating the other CPUs as far as APIC control? nothing really, when an AP starts it: enables LVT2 an NMI clears and sets its Task Priority Register to 0x10 enables its local APIC with 'focus prio' enabled. the BSP has been running with the same state since boot. it is managed 'assumming' there are other CPUs available, specifically: the IO APIC is sending messages in broadcast loprio mode, even though only the BSP is around to respond. when a CPU gets the mp_lock it lowers its TPR to 0x00, hoping to steer INTs to itself. I am assumming this works when the CPU has the lock for a syscall, but obviously won't help when it is servicing an INT as its PPR will override (but thats another issue). one difference is that when smp_active == 0, the get/rel lock calls return immediately so there is no adjustment of the TPR, but I cant concoct any scenario where that hurts anything. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Fri Dec 6 10:02:26 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA02878 for smp-outgoing; Fri, 6 Dec 1996 10:02:26 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id KAA02841 for ; Fri, 6 Dec 1996 10:02:20 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by who.cdrom.com (8.7.5/8.6.11) with SMTP id JAA05074 for ; Fri, 6 Dec 1996 09:31:18 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA20846; Fri, 6 Dec 1996 10:31:00 -0700 Message-Id: <199612061731.KAA20846@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Peter Wemm , smp@freebsd.org Subject: Re: last major problem In-reply-to: Your message of "Thu, 05 Dec 1996 23:47:35 MST." <199612060647.XAA18003@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 06 Dec 1996 10:31:00 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, Erich also reports his quad P6 failing after starting additional CPUs, in the same repeatable manner as everyone else. He reports ability to recompile kernel many times with only 1 CPU running. So I think its definately a P6 problem. Nobody running P5s have reported it. So the next question is whether its strictly an APIC_IO bug, an SMP_INVLTLB bug, or a bug from combining the 2. Since you can't run SMP_INVLTLB without APIO_IO, the one test people with P6s can do is to see if its any more stable with APIC_IO alone, ie. NO SMP_INVLTLB. If they are more (completely) stable without SMP_INVLTLB then it suggests that is the cause (but is NOT proof). If however the problem continues to exist then we completely eliminate SMP_INVLTLB as the culprit and can blame APIC_IO. P6 testers? (boy I'm jealous, wish I had one of those toys!) -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Fri Dec 6 10:02:40 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA02913 for smp-outgoing; Fri, 6 Dec 1996 10:02:40 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id KAA02898 for ; Fri, 6 Dec 1996 10:02:34 -0800 (PST) Received: from ceylon.informatik.uni-rostock.de (ceylon.informatik.uni-rostock.de [139.30.5.237]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id JAA05031 for ; Fri, 6 Dec 1996 09:22:47 -0800 (PST) Received: by ceylon.informatik.uni-rostock.de id SAA04001; Fri, 6 Dec 1996 18:22:42 +0100 Received: by donau.informatik.uni-rostock.de id SAA12630; Fri, 6 Dec 1996 18:22:40 +0100 (MET) Date: Fri, 6 Dec 1996 18:22:40 +0100 (MET) From: Gunther Hipper Message-Id: <199612061722.SAA12630@donau.informatik.uni-rostock.de> To: smp@csn.net Subject: Re: Great ! Cc: freebsd-smp@freebsd.org X-Sun-Charset: US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi all, hi Steve ! Hope it is not too late for an answer.. okay, I didn't cvsup since the last mail. Now: > > This is great news for me - since 19:50 MET the SMP kernel is > > __UP__ and __RUNNING__ ! No more IDE-problems ! > > Important: for the record, do you have APIC_IO and IDE working? > Okay, until now APIC_IO is __disabled__ in the config file. Here is (maybe) s.th. which might be interesting for you... I boot. Then, I turn SMP on: root@fcna1> sysctl -a|grep smp ; see what's on kern.smp_active: 0 root@fcna1> sysctl -w kern.smp_active=2 ; turn 2nd CPU on kern.smp_active: 0 -> 2 fcna1:{root} /usr/src/sys-SMP/i386/conf [113] vi SMP ; (this is not the interesting thing..) Segmentation fault (core dumped) fcna1:{root} /usr/src/sys-SMP/i386/conf [115] sysctl -w kern.smp_active=0 ;  ; Never returns.. But if you use another terminal, and set sysctl -w kern.smp_active=1 everything is fine, the sysctl -w kern.smp_active=0 finishes. vi, reboot remains at segmentation fault, i'll have to check the source (maybe make world again). I suppose this is no problem (who's interested in switching back) but maybe you want to know. Now - I'm changing to APIC_IO __enabled__. As to my knowledge - nothing changed in the smptests.h (#define TEST_LOPRIO, #define APIC_LAZY are turned on). Kernel boots fine, then: root@fcna1> sysctl -a|grep smp ; see what's on kern.smp_active: 0 root@fcna1> sysctl -w kern.smp_active=2 ; turn 2nd CPU on kern.smp_active: 0 -> 2 root@fcna1> vi ; runs fine root@fcna1> cd /usr/src/sys-SMP/i386/conf Segmentation fault (core dumped) root@fcna1> pwd /bin/pwd: Exec format error. Wrong Architecture. sysctl -w kern.smp_active=1 root@fcna1> kern.smp_active: 2 -> 1 root@fcna1> pwd /bin/pwd: Exec format error. Wrong Architecture. root@fcna1> ls Bus error (core dumped) root@fcna1> sysctl -w kern.smp_active=0 kern.smp_active: 1 -> 0 root@fcna1> ls Bus error (core dumped) fcna1:{root} [116] pwd /bin/pwd: Exec format error. Wrong Architecture. root@fcna1> reboot ; runs fine I'll now sup the latest cvs commits and try for the next time (starting to find out why reboot won't run). Please, ignore this mail if I was too long out of the group. I just thought you maybe interested in this.. Bye Gunther Gunther Hipper | University of Rostock | Department of Computer Science | Tel +49 381 498 3391 Albert-Einstein-Str. 21 | Fax +49 381 498 3440 18051 Rostock - Germany | Email Gunther.Hipper@informatik.uni-rostock.de From owner-freebsd-smp Fri Dec 6 10:02:49 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA02932 for smp-outgoing; Fri, 6 Dec 1996 10:02:49 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id KAA02915 for ; Fri, 6 Dec 1996 10:02:41 -0800 (PST) Received: from pat.idt.unit.no (pat.idt.unit.no [129.241.103.5]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id JAA05058 for ; Fri, 6 Dec 1996 09:28:48 -0800 (PST) Received: from idt.unit.no (tegge@ikke.idt.unit.no [129.241.111.65]) by pat.idt.unit.no (8.8.4/8.8.4) with ESMTP id SAA22564; Fri, 6 Dec 1996 18:28:05 +0100 (MET) Message-Id: <199612061728.SAA22564@pat.idt.unit.no> To: smp@bluenose.na.tuns.ca Cc: peter@spinner.dialix.com, smp@freebsd.org Subject: More info about fatal trap 12 In-Reply-To: Your message of "Fri, 6 Dec 1996 07:16:19 -0400 (AST)" References: <199612061116.HAA25054@bluenose.na.tuns.ca> X-Mailer: Mew version 1.06 on Emacs 19.33.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 06 Dec 1996 18:28:05 +0100 From: Tor Egge Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > For compiling a kernel, `make depend' crashes the system. > If 'make depend' is executed before `sysctl -w kern.smp_active=2', > the kernel can be compiled to the very end until > > ------------- > cc -c -x assembler-with-cpp -DLOCORE -nostdinc -I- -I. -I../.. -I../../../include -DFAILSAFE -DCOMPAT_43 -DCD9660 -DMSDOSFS -DNFS -DFFS -DINET -DKERNEL ../../i386/i386/swtch.s > --- vers.o --- > sh ../../conf/newvers.sh SMP -DFAILSAFE -DCOMPAT_43 -DCD9660 -DMSDOSFS -DNFS -DFFS -DINET > > :-( > ------------- > > I can provide the ktrace file, if you need it to identify the problem. > A closer examination of the kernel dump shows that the first page fault is from the user process /bin/sh. The call stack is #0 boot (howto=256) at ../../kern/kern_shutdown.c:267 #1 0xe0112d69 in panic (fmt=0xe01bcd3f "page fault") at ../../kern/kern_shutdown.c:395 #2 0xe01bda25 in trap_fatal (frame=0xdfbffe58) at ../../i386/i386/trap.c:747 #3 0xe01bd458 in trap_pfault (frame=0xdfbffe58, usermode=0) at ../../i386/i386/trap.c:654 #4 0xe01bd08b in trap (frame={tf_es = -270925808, tf_ds = 16, tf_edi = -270878108, tf_esi = -541077504, tf_ebp = -541065552, tf_isp = -541065600, tf_ebx = 89759744, tf_edx = -4194304, tf_ecx = -528396, tf_eax = -528396, tf_trapno = 12, tf_err = 0, tf_eip = -535058509, tf_cs = 8, tf_eflags = 66178, tf_esp = -532455933, tf_ss = -270885632}) at ../../i386/i386/trap.c:313 #5 0xe01ba7b3 in pmap_enter (pmap=0xefdaba64, va=3753889792, pa=89759744, prot=7 '\a', wired=0) at ../../i386/i386/pmap.c:2017 #6 0xe01a4193 in vm_fault (map=0xefdaba00, vaddr=3753889792, fault_type=3 '\003', change_wiring=0) at ../../vm/vm_fault.c:773 #7 0xe01bd3b0 in trap_pfault (frame=0xdfbfffbc, usermode=1) at ../../i386/i386/trap.c:634 #8 0xe01bcf33 in trap (frame={tf_es = 39, tf_ds = 39, tf_edi = 352256, tf_esi = 330220, tf_ebp = -541074424, tf_isp = -541065244, tf_ebx = 0, tf_edx = 1, tf_ecx = 330220, tf_eax = 0, tf_trapno = 12, tf_err = 7, tf_eip = 45296, tf_cs = 31, tf_eflags = 66050, tf_esp = -541074448, tf_ss = 39}) at ../../i386/i386/trap.c:241 ------ userland ------ /bin/sh ---- #9 0xb0f0 in ?? () [forkshell] #10 0x63ab in ?? () [evalcommand] #11 0x58e1 in ?? () [evaltree] #12 0xc11f in ?? () [cmdloop] #13 0xc02e in ?? () [main] #14 0x107e in ?? () [start] Dump of assembler code for function forkshell: 0xb0cc : pushl %ebp 0xb0cd : movl %esp,%ebp 0xb0cf : subl $0xc,%esp 0xb0d2 : pushl %edi 0xb0d3 : pushl %esi 0xb0d4 : pushl %ebx 0xb0d5 : movl 0x8(%ebp),%edi 0xb0d8 : movl 0xc(%ebp),%esi 0xb0db : movl 0x52898,%eax 0xb0e0 : incl %eax 0xb0e1 : movl %eax,0x52898 0xb0e6 : movl 0x52898,%eax 0xb0eb : call 0x286c8 0xb0f0 : movl %eax,0xfffffff8(%ebp) 0xb0f3 : cmpl $0xffffffff,%eax The first access to the stack by the child process failed when trying to save the return value from fork. The parent process was running on CPU #1, and the child process was running on CPU #0. - Tor Egge From owner-freebsd-smp Fri Dec 6 10:15:37 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA04008 for smp-outgoing; Fri, 6 Dec 1996 10:15:37 -0800 (PST) Received: from spinner.DIALix.COM (root@spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id KAA04001 for ; Fri, 6 Dec 1996 10:15:32 -0800 (PST) Received: from spinner.DIALix.COM (peter@localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id CAA19205; Sat, 7 Dec 1996 02:15:13 +0800 (WST) Message-Id: <199612061815.CAA19205@spinner.DIALix.COM> To: Tor Egge cc: smp@bluenose.na.tuns.ca, smp@freebsd.org Subject: Re: More info about fatal trap 12 In-reply-to: Your message of "Fri, 06 Dec 1996 18:28:05 +0100." <199612061728.SAA22564@pat.idt.unit.no> Date: Sat, 07 Dec 1996 02:15:12 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Tor Egge wrote: > A closer examination of the kernel dump shows that the first page fault > is from the user process /bin/sh. The call stack is [..] > The first access to the stack by the child process failed when trying > to save the return value from fork. > > The parent process was running on CPU #1, and the child process > was running on CPU #0. > > - Tor Egge Hmm!! The plot thickens! I noticed the failing pmap_enter was at 0xefbfd000 which is the first stack page already, but I wasn't sure if it was the initial creation, or if the stack had been paged out and was failing on pagein. Thank you! Confirmation of this is MOST useful! I had redone the fork implementation some time ago to support address space sharing for kernel supported user level threads. I have some bad memories about this code.. In fact, I have an idea that deserves a closer look... Cheers, -Peter From owner-freebsd-smp Fri Dec 6 10:24:06 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA04341 for smp-outgoing; Fri, 6 Dec 1996 10:24:06 -0800 (PST) Received: from pat.idt.unit.no (0@pat.idt.unit.no [129.241.103.5]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id KAA04317 for ; Fri, 6 Dec 1996 10:24:00 -0800 (PST) Received: from idt.unit.no (tegge@ikke.idt.unit.no [129.241.111.65]) by pat.idt.unit.no (8.8.4/8.8.4) with ESMTP id TAA24340; Fri, 6 Dec 1996 19:23:22 +0100 (MET) Message-Id: <199612061823.TAA24340@pat.idt.unit.no> To: peter@spinner.dialix.com Cc: smp@bluenose.na.tuns.ca, smp@freebsd.org Subject: Re: More info about fatal trap 12 In-Reply-To: Your message of "Sat, 07 Dec 1996 02:15:12 +0800" References: <199612061815.CAA19205@spinner.DIALix.COM> X-Mailer: Mew version 1.06 on Emacs 19.33.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Fri, 06 Dec 1996 19:23:21 +0100 From: Tor Egge Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Tor Egge wrote: > > A closer examination of the kernel dump shows that the first page fault > > is from the user process /bin/sh. The call stack is > [..] > > The first access to the stack by the child process failed when trying > > to save the return value from fork. > > > > The parent process was running on CPU #1, and the child process > > was running on CPU #0. > > > > - Tor Egge > > Hmm!! The plot thickens! I noticed the failing pmap_enter was at > 0xefbfd000 which is the first stack page already, but I wasn't sure > if it was the initial creation, or if the stack had been paged out > and was failing on pagein. No pageouts. (kgdb) print/x cnt.v_free_count $20 = 0x1b607 - Tor Egge From owner-freebsd-smp Fri Dec 6 11:10:00 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA06398 for smp-outgoing; Fri, 6 Dec 1996 11:10:00 -0800 (PST) Received: (from fsmp@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA06391 for freebsd-smp; Fri, 6 Dec 1996 11:09:58 -0800 (PST) Date: Fri, 6 Dec 1996 11:09:58 -0800 (PST) From: Steve Passe Message-Id: <199612061909.LAA06391@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/kern init_smp.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk fsmp 96/12/06 11:09:58 Modified: kern init_smp.c Log: added "#error ..." line forcing people to be aware of SMP_AUTOSTART problem. Revision Changes Path 1.44 +2 -1 sys/kern/init_smp.c From owner-freebsd-smp Fri Dec 6 11:11:08 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA03408 for smp-outgoing; Fri, 6 Dec 1996 10:04:35 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id KAA03383 for ; Fri, 6 Dec 1996 10:04:25 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by who.cdrom.com (8.7.5/8.6.11) with SMTP id IAA04703 for ; Fri, 6 Dec 1996 08:27:13 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vW3AH-0007kq-00; Fri, 6 Dec 1996 08:30:17 -0800 To: Steve Passe , Peter Wemm cc: smp@freebsd.org Subject: P6 and FreeBSD/SMP (was -> Re: last major problem) In-reply-to: Your message of "Thu, 05 Dec 1996 23:47:35 MST." <199612060647.XAA18003@clem.systemsix.com> Date: Fri, 06 Dec 1996 08:30:17 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve Passe writes: > > Steve Passe wrote: > > > Hi, > > > > > > so all 3 failing systems are P6. is there anyone now successfully running > > > APIC_IO on a P6 system? > > > > Wait a sec, I thought this was a problem caused by the SMP_INVLTLB code? > > Or is it a generic "P6 dislikes APIC_IO in general" problem? > > thats what I'm wondering, I've been so busy fighting the other fires that > I haven't kept up on the details of this and may misunderstand... > > so additionally, has ANYONE EVER run a 'solid' APIC_IO kernel on a P6? Hmmm... I think there are several issues afoot here. (1) Does the P6 run an APIC_IO kernel OK before activating the other CPUs (2) Does the P6 run an APIC_IO kernel OK after activating the other CPUs ...and seen in another message... (3) [the thing about relying on the short prefect queue of the Pentium, and the P6 might break it] I ran a bunch of tests last night and this morning with the kernel tree from yesterday afternoon/evening. My results were (using APIC_IO + SMP_INVLTLB) : -- After I activate the other 3 CPUs (via "sysctl -w kern.smp_active=4"), standard (sinple) operations seem OK, but when I start compiling a kernel, anywhere from 1/3 to 1/2 the way through it dies with a "kernel trap 12: supervisor read/write, page not present" break to the kernel debugger, going into pmap_enter. It is always that error (I think I've seen a "write" once, with it saying a "read" trap the other times). This implies that the answer to #2 is "no" (at least on my test box). I tried this about a dozen times, with the same results each time. -- If I don't activate the other CPUs, I can do a dozen builds in a raw with no problems. This implies the answwer to #1 is "yes". This leads me to believe that the problem is in the MP handling, not the base APIC_IO stuff. Maybe this is a good time to tell some of us what is different when activating the other CPUs as far as APIC control? I've been digging through the source, and it seems that the "smp_invltlb" is separate from the normal "invltlb" function? There are *many* places where "invltlb_1pg" (I think that was it) or other variants are called and no SMP invalidates are propagated. This strikes me as a situation fraught with potential (and currently real) problems. What is the design goal here? On a side note (issue #3 I saw a comment on), first of all, never, *ever* rely on short prefetch queues. A proper sequence which flushes the queues in the appropriate places should be used. The Intel manuals have many examples of this. I'll take a look at the code sequence and see what's up with it. It could be that the comment that was made about it was bogus. (if not, then that could very well be a fatal problem. Certainly the P6 and what I've heard of the next project will become more and more unpredictable unless the proper methods to serialize control register changes are used). -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Fri Dec 6 11:46:04 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA07832 for smp-outgoing; Fri, 6 Dec 1996 11:46:04 -0800 (PST) Received: from cs.utah.edu (cs.utah.edu [128.110.4.21]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id LAA07827 for ; Fri, 6 Dec 1996 11:46:02 -0800 (PST) Received: from fast.cs.utah.edu by cs.utah.edu (8.8.3/utah-2.21-cs) id MAA07175; Fri, 6 Dec 1996 12:45:55 -0700 (MST) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id MAA17245; Fri, 6 Dec 1996 12:45:54 -0700 Date: Fri, 6 Dec 1996 12:45:54 -0700 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199612061945.MAA17245@fast.cs.utah.edu> To: peter@spinner.dialix.com, smp@csn.net, smp@freebsd.org Subject: Re: last major problem Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On a related note (to the P6 not working, and the Pentium working), the PCI Cyclades driver (in 2.1.5, with patches from DG) works on a Pentium as well, but not on a P6 (I think it was a reserved trap; it's been a while). DG said there were similar problems on wcarchive (P6 related?) Kevin From owner-freebsd-smp Fri Dec 6 14:02:10 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id OAA13190 for smp-outgoing; Fri, 6 Dec 1996 14:02:10 -0800 (PST) Received: from uruk.org (root@ns.uruk.org [198.145.95.253]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id OAA13183 for ; Fri, 6 Dec 1996 14:02:05 -0800 (PST) Received: from uruk.org [127.0.0.1] (erich) by uruk.org with esmtp (Exim 0.53 #1) id E0vW8OA-0008PE-00; Fri, 6 Dec 1996 14:04:58 -0800 To: Peter Wemm cc: smp@csn.net, smp@freebsd.org Subject: Re: P6 and FreeBSD/SMP (was -> Re: last major problem) In-reply-to: Your message of "Sat, 07 Dec 1996 01:36:44 +0800." <199612061736.BAA18860@spinner.DIALix.COM> Date: Fri, 06 Dec 1996 14:04:58 -0800 From: Erich Boleyn Message-Id: Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Peter Wemm writes: ... > > "kernel trap 12: supervisor read/write, page not present" break to > > the kernel debugger, going into pmap_enter. It is always that error > > (I think I've seen a "write" once, with it saying a "read" trap the > > other times). This implies that the answer to #2 is "no" (at least > > on my test box). I tried this about a dozen times, with the same > > results each time. > > Question: what are the offending faulting addresses? We use two PTD slots, > one for accessing the "current" process (addresses 0xefc00000-0xeffeffff), > and the other being for an "alternate" process or address space (range: > 0xffc00000 - 0xfffeffff). These 4MB chunks are a sparse end-to-end set > of page table pages representing the 4GB of process address space. I don't have many available offhand, but I do remember they were always in the second (0xffc00000 - 0xfffeffff) range you listed. The one I have offhand was at 0xffc00144, for "sh". I'll try generating some more tonight. Maybe a more complete look at the other parameters present too. > > -- If I don't activate the other CPUs, I can do a dozen builds in a raw > > with no problems. This implies the answwer to #1 is "yes". > > How does it go without SMP_INVLTLB BTW? Do you use a scsi or ide system? > I think we've pretty much discovered that the IDE driver is very vulnerable > to missing the invltlb calls on the alternate cpu's for some reason. My test system is on a SCSI disk (adaptec 2940 PCI, or more accurately a motherboard 7870 chip on a part of the PCI bus). I won't be able to test on IDE for a little while. I could test without SMP_INVLTLB, but I'm quite leery of simply shutting off the only way of propagating invalidates, which will probably cause as many problems as it solves (earlier on, I could never get kernel compiles to run very far anyway... many wierd things would happen, from segfaults to system hangs... I much prefer dropping into the kernel debugger where in theory I can look for something). > smp_invltlb() is called from the invltlb() function. When compiling > with SMP_INVLTLB, invltlb() is no longer inlined.. it's a called > function in mp_machdep.c > > Yes, this is extreme overkill, probably 90% of those calls to smp_invltlb() > are unnecessary.. They should not be harmful, but once it's working we can > optimise it quite a lot. Yes, I very much think getting it to work right in the first place is the most important thing. Question: is "smp_invltlb()" called from all the variants of "invltlb()", including the 1-page version? ...[my panic deleted]... > urk, sorry if I gave the impression that we were deliberately doing this.. > No, it was an *accident* that the early smp code did this, it worked fine > on the P5, but failed on the P6. I was thinking out aloud to the effect > of "Hmm, I wonder if there's somewhere else that this kind of thing is > happening that we're not aware of?". OK. Sorry about the panic here. You're saying this was already dealt with, so good enough. -- Erich Stefan Boleyn \_ E-mail (preferred): Mad Genius wanna-be, CyberMuffin \__ (finger me for other stats) Web: http://www.uruk.org/~erich/ Motto: "I'll live forever or die trying" From owner-freebsd-smp Fri Dec 6 14:53:06 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id OAA15756 for smp-outgoing; Fri, 6 Dec 1996 14:53:06 -0800 (PST) Received: (from fsmp@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id OAA15748 for freebsd-smp; Fri, 6 Dec 1996 14:53:05 -0800 (PST) Date: Fri, 6 Dec 1996 14:53:05 -0800 (PST) From: Steve Passe Message-Id: <199612062253.OAA15748@freefall.freebsd.org> To: freebsd-smp Subject: cvs commit: sys/i386/i386 machdep.c mp_machdep.c mpapic.c Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk fsmp 96/12/06 14:53:04 Modified: i386/i386 machdep.c mp_machdep.c mpapic.c Log: butchered my beautifully formatted code in the name of 'style'! Revision Changes Path 1.34 +6 -9 sys/i386/i386/machdep.c 1.33 +200 -199 sys/i386/i386/mp_machdep.c 1.24 +68 -64 sys/i386/i386/mpapic.c From owner-freebsd-smp Fri Dec 6 21:27:45 1996 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id VAA04702 for smp-outgoing; Fri, 6 Dec 1996 21:27:45 -0800 (PST) Received: from trapdoor.aracnet.com (root@trapdoor.aracnet.com [204.188.47.1]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id VAA04693 for ; Fri, 6 Dec 1996 21:27:42 -0800 (PST) Received: from cbrowni2-home (ppp-h8.aracnet.com [204.188.47.201]) by trapdoor.aracnet.com (8.7.4/8.6.9) with SMTP id VAA30022; Fri, 6 Dec 1996 21:27:33 -0800 Message-ID: <32A8FFD0.2A72@aracnet.com> Date: Fri, 06 Dec 1996 21:25:36 -0800 From: Chris Browning Reply-To: cbrown@aracnet.com X-Mailer: Mozilla 3.01 (WinNT; U) MIME-Version: 1.0 To: Steve Passe CC: smp@freebsd.org Subject: Re: dual-cpu PPRO motherboards... References: <199610161850.MAA16302@clem.systemsix.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Steve, Here is some of the info about my machine. Sorry it took so long to get back to you. Chris > This terminology can get very confusing at times, can't it! > > >Actually, I am talking about the APIC bus spec. It supports 255 > >interrupt > >sources. The P6 (I'm not sure about the P5) atleast supports this. > >When you say IP APIC, which one are you talking about? > ^^ > I assume you meant IO here? The only one I have encountered is the > 82093AA IO APIC. There is (was) also the 82489DX APIC which could be used > as both a "local" APIC with 80486 CPUs and as an IO APIC. But Intel doesn't > make these anymore, at least not for new systems (there may be replacement > parts available). > > > I know of atleast > >3 that are commonly found on systems, probably more. Each of them > >has a different number of interrupts and capabilies. Not to mention > >systems with more than one IO APIC. > > what other parts do you know of (numbers please!) Here is the information off the top of the chips. The first and the last one are responsible for routing interrupts in the system. (M)AMI 9511ATY 6721-502 Intel (C) 94 620368-001 Intel(R) PCIset S82375SB S Z955 16A8EF1070 JAPAN9527EAI (C)'92008 Intel(R) PCIset S82374SB S Z892 16T6F1004 JAPAN9520EAI (C)'92037 > > What do you need to know? The machine I am tying this message on right > >now has > >2 IO APICs in it. Neither are PIIX/PIIX3. > > The output of 'mptable', with both IO APICs enabled in the BIOS. This > will show me how the variuos INTerrupt sources are associated with > IO APIC pins, among other useful data. > You can find this program on the SMP web page at the top of the > "Various tests that you can help us run:" section: > > http://www.freebsd.org/~fsmp/SMP/SMP.html > > This program can be run without the SMP kernel being active, but the 2nd > IO APIC MUST be enabled in the BIOS for its assignments to show up. > Additionally, if this is an EISA machine it may or may not work, depending > on where they stuck the MP table. Well, I couldn't get this to compile on Linux, which is what is loaded on the machine at this time. It doesn't have the debug kernal either. Sorry. > Also, give me your 'dmesg' output, as well as the part #s of your IO APICs > and chipset if known. Here is the dmesg output.... (s): fd0 is 1.44M Started kswapd v 1.4.2.2 FDC 0 is an 82078. aic7xxx: BurstLen = 8 DWDs, Latency Timer = 96 PCLKS aic7xxx: devconfig = 0x580. aic7xxx: Reading SEEPROM... aic7xxx: Unable to read SEEPROM; using leftover BIOS values. aic7xxx: Extended translation disabled. aic7xxx: Using 16 SCB's after checking for SCB memory. aic7xxx: Enabling wide channel of AIC-7870-Wide. AIC-7870-WIDE (PCI-bus): irq 10 bus release time 40 bclks data fifo threshold 100% SCSI CHANNEL A: scsi id 7 scsi selection timeout 256 ms scsi bus reset at power-on enabled scsi bus parity enabled aic7xxx: Downloading sequencer code...done. aic7xxx: Resetting the SCSI bus...done. aic7xxx: BurstLen = 8 DWDs, Latency Timer = 96 PCLKS aic7xxx: devconfig = 0x580. aic7xxx: Reading SEEPROM... aic7xxx: Unable to read SEEPROM; using leftover BIOS values. aic7xxx: Extended translation disabled. aic7xxx: Using 16 SCB's after checking for SCB memory. aic7xxx: Enabling wide channel of AIC-7870-Wide. AIC-7870-WIDE (PCI-bus): irq 5 bus release time 40 bclks data fifo threshold 100% SCSI CHANNEL A: scsi id 7 scsi selection timeout 256 ms scsi bus reset at power-on enabled scsi bus parity enabled aic7xxx: Downloading sequencer code...done. aic7xxx: Resetting the SCSI bus...done. scsi0 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 3.2/3.1/3.0 scsi1 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 3.2/3.1/3.0 scsi : 2 hosts. aic7xxx: Scanning channel A for devices. aic7xxx: Target 0, channel A, now synchronous at 10.0MHz, offset(0xf). Vendor: SEAGATE Model: ST31230N Rev: 0510 Type: Direct-Access ANSI SCSI revision: 02 Detected scsi disk sda at scsi0, channel 0, id 0, lun 0 aic7xxx: Scanning channel A for devices. scsi : detected 1 SCSI disk total. SCSI device sda: hdwr sector= 512 bytes. Sectors= 2069860 [1010 MB] [1.0 GB] Partition check: sda: sda1 sda2 hda: hda1 hda2 < hda5 hda6 > VFS: Mounted root (ext2 filesystem) readonly. lp: Driver configured but no interfaces found. CSLIP: code copyright 1989 Regents of the University of California SLIP: version 0.8.4-NET3.019-NEWTTY-MODULAR (dynamic channels, max=256). SLIP linefill/keepalive option. PPP: version 2.2.0 (dynamic channel allocation) PPP Dynamic channel allocation code copyright 1995 Caldera, Inc. PPP line discipline registered. hdb : tray open or drive not ready hdb : tray open or drive not ready hdb : tray open or drive not ready VFS: Disk change detected on device 03:40 hdb : tray open or drive not ready hdb : tray open end_request: I/O error, dev 03:40, sector 65 isofs_read_super: bread failed, dev 03:40 iso_blknum 16 block 32 hdb : tray open or drive not ready hdb : tray open or drive not ready hdb : tray open or drive not ready VFS: Disk change detected on device 03:40 hdb : tray open or drive not ready hdb : tray open end_request: I/O error, dev 03:40, sector 65 isofs_read_super: bread failed, dev 03:40 iso_blknum 16 block 32 Unable to identify CD-ROM format. eepro100.c:v0.12b 6/18/96 Donald Becker becker@cesdis.gsfc.nasa.gov Found Intel i82557 PCI Speedo at I/O 0xf4e0, IRQ 9. PCI latency timer (CFLT) is 0x60. eepro100.c:v0.12b 6/18/96 Donald Becker becker@cesdis.gsfc.nasa.gov eth0: Intel EtherExpress Pro 10/100 at 0xf4e0, 00:20:35:22:BB:C6, IRQ 9. Board assembly 645345-001, Physical connectors present: RJ45 Primary interface chip DP83840 PHY #1. DP83840 specific setup... General self-test: passed. Serial sub-system self-test: passed. Internal registers self-test: passed. ROM checksum self-test: passed (0x49caa8d6). eth0: speedo_open() irq 9. eth0: Done speedo_open(), status 00000090. eth0: Media selection tick, status 0050. iBCS: socksys registered on character major 30 ================================================================= And here is a selected list of proc outputs Text item: pci.txt 11/22/96 2:18P PCI devices found: Bus 0, device 26, function 0: Host bridge: Intel Orion P6 (rev 2). Medium devsel. Fast back-to-back capable. Master Capable. Latency=72. Bus 0, device 25, function 0: Host bridge: Intel Orion P6 (rev 2). Medium devsel. Fast back-to-back capable. Master Capable. Latency=72. Bus 0, device 15, function 0: Unknown class: Intel Unknown device (rev 0). Vendor id=8086. Device id=8. Fast devsel. Fast back-to-back capable. IRQ 255. Prefetchable 32 bit memory at 0xfec01000. Bus 0, device 14, function 0: Non-VGA device: Intel 82375EB (rev 5). Medium devsel. Master Capable. Latency=72. Bus 0, device 12, function 0: VGA compatible controller: Weitek P9100 (rev 0). Medium devsel. IRQ 11. Non-prefetchable 32 bit memory at 0xfd000000. Text item: ioports.txt 11/22/96 2:18P 0000-001f : dma1 0020-003f : pic1 0040-005f : timer 0060-006f : keyboard 0070-007f : rtc 0080-009f : dma page reg 00a0-00bf : pic2 00c0-00df : dma2 00f0-00ff : npu 01f0-01f7 : ide0 03c0-03df : vga+ 03f0-03f5 : floppy 03f6-03f6 : ide0 03f7-03f7 : floppy DIR 03f8-03ff : serial(auto) f4e0-f4ff : Intel Speedo3 Ethernet 10400-104be : aic7xxx 10800-108be : aic7xxx Text item: ints.txt 11/22/96 2:18P 0: 130555 timer 1: 705 keyboard 2: 0 cascade 5: 30 + aic7xxx 8: 0 + rtc 9: 6929 Intel EtherExpress Pro 10/100 Ethernet 10: 36 + aic7xxx 13: 23047 + IPI 14: 13125 + ide0 Text item: cpu.txt 11/22/96 2:18P processor : 0 cpu : 686 model : Pentium Pro vendor_id : GenuineIntel stepping : 6 fdiv_bug : no hlt_bug : no fpu : yes fpu_exception : yes cpuid : yes wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmov bogomips : 199.47 processor : 2 cpu : 686 model : Pentium Pro vendor_id : GenuineIntel stepping : 6 fdiv_bug : no hlt_bug : no fpu : yes fpu_exception : yes cpuid : yes wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic 11 mtrr pge mca cmov bogomips : 199.88