From owner-freebsd-smp Sun Feb 2 03:37:10 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id DAA14638 for smp-outgoing; Sun, 2 Feb 1997 03:37:10 -0800 (PST) Received: from parkplace.cet.co.jp (parkplace.cet.co.jp [202.32.64.1]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id DAA14632 for ; Sun, 2 Feb 1997 03:37:06 -0800 (PST) Received: from localhost (michaelh@localhost) by parkplace.cet.co.jp (8.8.5/CET-v2.1) with SMTP id LAA02818; Sun, 2 Feb 1997 11:36:36 GMT Date: Sun, 2 Feb 1997 20:36:36 +0900 (JST) From: Michael Hancock To: netdev@roxanne.nuclecu.unam.mx, "David S. Miller" cc: roque@di.fc.ul.pt, freebsd-smp@freebsd.org Subject: Re: SMP In-Reply-To: <199701311231.HAA15221@jenolan.caipgeneral> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Fri, 31 Jan 1997, David S. Miller wrote: > step 2: > > Ok, no self locking technique exists for what you are > tackling. Ok, plan B. > > At this point, you are going to go after a traditional locking > technique. But make the most of it. The traditional way > people think about this stuff for performance is to try and > make it such that less people want to lock the same stuff. > > I tend to disagree with that philosophy for certain > situations, and instead I would incourage someone to take > the following approach first. > > "Try to find a locking strategy that encourages code which > holds the lock for _extremely_ short periods of time." > > That is, strive for short holds instead of less contention. > It almost sounds like there are cases where "short holds" and "less contention" are hard to achieve. Can you give us an example? Or are you saying that spending time on contention minimization is not very fruitful. I think some key subsystems would benefit from per processor pools (less contention) and the code can still be structured for short holds. Regards, Mike Hancock From owner-freebsd-smp Sun Feb 2 04:02:57 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id EAA15346 for smp-outgoing; Sun, 2 Feb 1997 04:02:57 -0800 (PST) Received: from caipfs.rutgers.edu (root@caipfs.rutgers.edu [128.6.155.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id EAA15340 for ; Sun, 2 Feb 1997 04:02:54 -0800 (PST) Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id HAA27324; Sun, 2 Feb 1997 07:02:43 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id HAA09281; Sun, 2 Feb 1997 07:02:25 -0500 Date: Sun, 2 Feb 1997 07:02:25 -0500 Message-Id: <199702021202.HAA09281@jenolan.caipgeneral> From: "David S. Miller" To: michaelh@cet.co.jp CC: netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org In-reply-to: (message from Michael Hancock on Sun, 2 Feb 1997 20:36:36 +0900 (JST)) Subject: Re: SMP Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Date: Sun, 2 Feb 1997 20:36:36 +0900 (JST) From: Michael Hancock It almost sounds like there are cases where "short holds" and "less contention" are hard to achieve. Can you give us an example? Or are you saying that spending time on contention minimization is not very fruitful. It is hard to achieve in certain cirsumstances yes, but it is worth putting some effort towards just not "too much" if things begin to look a bit abismal. I think some key subsystems would benefit from per processor pools (less contention) and the code can still be structured for short holds. Agreed. This becomes a serious issue once you have parallelized a subsystem. Even though all cpu's can go about their own ways if they are messing with seperate object, all of them contend equally for the kernel memory allocator. As Terry Lambert pointed out, even if you have per-processor pools you run into many cache pulls between processors when you are on one cpu, allocate a resource, get scheduled on another and then free it from there. (the TLB invalidation cases he described are not an issue at all for us on the kernel side, brain damaged traditional unix kernel memory designs eat this overhead, we won't, other than vmalloc() chunks we eat no mapping setup and destruction overhead at all and thus nothing to keep "consistant" with IPI's since it is not changing) On the other hand, I like the idea of: 1) per cpu pools for object FOO 2) global secondard pool for object class BAR which FOO is a member of and thus the strategy becomes: alloc_a_FOO() { again: if(FOO_free_list[smp_processor_id()]) { /* Go like smoke */ } else { lock_global_pool(); get_some_for_me(); if(others_running_low) feed_them_too(); unlock_global_pool(); goto again; } } free_a_FOO() { if(my_pool_only_moderately_full) { give_it_to_me(); } else { feed_someone_else_who_needs_it(); } } The idea is that when you need to enter a contension situation, you make the most of it by working towards keeping others from running into the same situation. Another modification to the idea on the freeing side is to always free this FOO to the local per-cpu queue, and if other cpus could starve soon you give some of your oldest FOO's to them, the encourage cache locality. Just some ideas. These are some of ideas I was toying about with Larry McVoy when we were discussing with each other how N processors could go full throttle on an N high speed interface machine with near zero contention. ---------------------------------------------//// Yow! 11.26 MB/s remote host TCP bandwidth & //// 199 usec remote TCP latency over 100Mb/s //// ethernet. Beat that! //// -----------------------------------------////__________ o David S. Miller, davem@caip.rutgers.edu /_____________/ / // /_/ >< From owner-freebsd-smp Sun Feb 2 08:23:44 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id IAA24005 for smp-outgoing; Sun, 2 Feb 1997 08:23:44 -0800 (PST) Received: from smerdon.livonia.mi.us (root@pm143-17.dialip.mich.net [198.110.144.27]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id IAA24000 for ; Sun, 2 Feb 1997 08:23:32 -0800 (PST) Received: from p133 (e0.i386.smerdon.livonia.mi.us [199.33.147.37]) by smerdon.livonia.mi.us (8.7.5/8.6.9) with SMTP id LAA21631; Sun, 2 Feb 1997 11:23:13 -0500 (EST) Message-Id: <3.0.32.19970202112303.00908e80@smerdon.livonia.mi.us> X-Sender: jds@smerdon.livonia.mi.us X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Sun, 02 Feb 1997 11:23:05 -0500 To: Steve Passe From: "John D. Smerdon" Subject: Re: Tyan Tomcat II SMP video problems Cc: smp@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk At 09:58 PM 2/1/97 -0700, Steve Passe wrote: >so the question is why a kernel with APIC_IO *and* SMP_INVLTLB doesn't... >can you verify that this is true, ie have you tried that combination? That was the first one I tried. I tried it again with the ep0 on irq 12 and the same things happened. I also just noticed that the time stops. /var/log/messages has everything at the same time and a `date` keeps returning the same time. The system boots normally until the "SMP: All idle procs online." message is displayed. The system appears to hang, but is really running and not updating the video. I can blindly type and login or telnet in from another system. Enabling the second processor causes the screen to display other messages in the booting sequence, but then the video seems to hang again. The telnet session continues to work. Doing a `shutdown -r now` never completes, although the telnet session is still active. >could you document for us how one determines whether they have the "new and >improved" module? serial #, model #, ??? The FAQ at the Tyan site (http://www.tyan.com/html/tomcat.html) lists the correct model numbers as: 256K Module S1543-PB256 512K Module S1543-PB512A The Tomcat III has the cache soldered on the board. >this was a shot in the dark. I seem to remember someone claiming that tyan >had a problem with reassigning 14 & 15 as they are "reserved" for the 2 IDE >channels. since the IDE circuits are in the same chip as the 8259 ICUs >it is possible that there are some limitations, but I have no concrete >facts either way. The manual lists irq 14 for the "Hard Disk Adapter" and 9-12, and 15 as available. The Matrox and the 2940 now share irq 11 when booting with the APIC_IO/SMP_INVLTLB disabled. Here is the mptable output with APIC_IO/SMP_INVLTLB enabled: ============================================================================ === MPTable, version 2.0.6 looking for EBDA pointer @ 0x040e, NOT found searching CMOS 'top of mem' @ 0x0009fc00 (639K) searching BIOS @ 0x000f0000 MP FPS found in BIOS @ physical addr: 0x000f0c80 ---------------------------------------------------------------------------- --- MP Floating Pointer Structure: location: BIOS physical address: 0x000f0c80 signature: '_MP_' length: 16 bytes version: 1.1 checksum: 0xf4 mode: Virtual Wire ---------------------------------------------------------------------------- --- MP Config Table Header: physical address: 0x000f0c94 signature: 'PCMP' base table length: 292 version: 1.1 checksum: 0xa5 OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 28 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ---------------------------------------------------------------------------- --- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 0 0x11 BSP, usable 5 2 1 0x07bf 1 0x11 AP, usable 5 2 1 0x07bf -- Bus: Bus ID Type 0 ISA 1 PCI -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT conforms conforms 0 0 2 0 INT conforms conforms 0 1 2 1 INT conforms conforms 0 0 2 2 INT conforms conforms 0 3 2 3 INT conforms conforms 0 4 2 4 INT conforms conforms 0 5 2 5 INT conforms conforms 0 6 2 6 INT conforms conforms 0 7 2 7 INT conforms conforms 0 8 2 8 INT conforms conforms 0 9 2 9 INT conforms conforms 0 10 2 10 INT conforms conforms 0 11 2 11 INT conforms conforms 0 12 2 12 INT conforms conforms 0 13 2 13 INT conforms conforms 0 14 2 14 INT conforms conforms 0 15 2 15 INT active-lo level 1 20:A 2 16 INT active-lo level 1 19:A 2 17 INT active-lo level 1 18:A 2 18 INT active-lo level 1 17:A 2 19 SMI conforms conforms 0 0 2 23 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT active-hi edge 0 0 255 0 NMI active-hi edge 0 0 255 1 ---------------------------------------------------------------------------- --- # SMP kernel config file options: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O options NCPU=2 # number of CPUs options NBUS=2 # number of busses options NAPIC=1 # number of IO APICs options NINTR=24 # number of INTs options SMP_INVLTLB # #options SMP_PRIVPAGES # BROKEN, DO NOT use! #options SMP_AUTOSTART # BROKEN, DO NOT use! #options SERIAL_DEBUG # com port debug output ---------------------------------------------------------------------------- --- dmesg output: Copyright (c) 1992-1996 FreeBSD Inc. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 3.0-SMP #0: Sun Feb 2 10:07:55 EST 1997 jds@p133.smerdon.livonia.mi.us:/usr/src/sys/compile/SMERDONSMPAPIC FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00030010 cpu1 (AP): apic id: 1, version: 0x00030010 io0 (APIC): apic id: 2, version: 0x00170011 Calibrating clock(s) relative to mc146818A clock ... i8254 clock: 1193120 Hz CPU: Pentium (586-class CPU) Origin = "GenuineIntel" Id = 0x52c Stepping=12 Features=0x3bf real memory = 67108864 (65536K bytes) avail memory = 63594496 (62104K bytes) Probing for devices on PCI bus 0: chip0 rev 2 on pci0:0 chip1 rev 1 on pci0:7:0 chip2 rev 0 on pci0:7:1 vga0 rev 1 int a irq 19 on pci0:17 Freeing (NOT implimented) irq 11 for ISA cards. ahc0 rev 0 int a irq 17 on pci0:19 Freeing (NOT implimented) irq 11 for ISA cards. ahc0: aic7880 Wide Channel, SCSI Id=7, 16 SCBs ahc0 waiting for scsi devices to settle (ahc0:0:0): "Quantum XP34300W L912" type 0 fixed SCSI 2 sd0(ahc0:0:0): Direct-Access 4101MB (8399520 512 byte sectors) ahc0:A:5: refuses WIDE negotiation. Using 8bit transfers (ahc0:5:0): "TOSHIBA CD-ROM XM-3701TA 0236" type 5 removable SCSI 2 cd0(ahc0:5:0): CD-ROM can't get the size Probing for devices on the ISA bus: sc0 at 0x60-0x6f irq 1 on motherboard sc0: VGA color <16 virtual consoles, flags=0x0> sio0 at 0x3f8-0x3ff irq 4 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A sio2 at 0x3e8-0x3ef irq 9 on isa sio2: type 16550A sio3: disabled, not probed. lpt0 at 0x378-0x37f irq 7 on isa lpt0: Interrupt-driven port lp0: TCP/IP capable interface fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: NEC 72065B fd0: 1.44MB 3.5in uha0 not found at 0x330 aha0 not found at 0x330 aic0 not found at 0x340 scd0 not found at 0x230 1 3C5x9 board(s) on ISA found at 0x300 ep0 at 0x300-0x30f irq 12 on isa ep0: aui/utp/bnc[*BNC*] address 00:a0:24:be:b8:c0 npx0 on motherboard npx0: INT 16 interface apm0: disabled, not probed. joy0 at 0x201 on isa joy0: joystick sb0 at 0x220 irq 5 drq 1 on isa sb0: sbxvi0 at 0x0 drq 5 on isa sbxvi0: sbmidi0 at 0x330 on isa changing root device to sd0a Enabled INTs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 17, imen: 0x00fdec01 SMP: All idle procs online. ============================================================================ === -- John D. Smerdon; Livonia, Michigan, USA; Contents are my opinion. Home: jds@smerdon.livonia.mi.us From owner-freebsd-smp Sun Feb 2 08:56:03 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id IAA25120 for smp-outgoing; Sun, 2 Feb 1997 08:56:03 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id IAA25103 for ; Sun, 2 Feb 1997 08:55:58 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id JAA08947; Sun, 2 Feb 1997 09:53:00 -0700 Message-Id: <199702021653.JAA08947@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: "John D. Smerdon" cc: smp@freebsd.org Subject: Re: Tyan Tomcat II SMP video problems In-reply-to: Your message of "Sun, 02 Feb 1997 11:23:05 EST." <3.0.32.19970202112303.00908e80@smerdon.livonia.mi.us> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 02 Feb 1997 09:53:00 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >>so the question is why a kernel with APIC_IO *and* SMP_INVLTLB doesn't... >>can you verify that this is true, ie have you tried that combination? > >That was the first one I tried. I tried it again with the ep0 on irq 12 >and the same things happened. > >I also just noticed that the time stops. /var/log/messages has everything >at the same time and a `date` keeps returning the same time. this is an important clue. do you have the screensaver enabled in the /etc/sysconfig file? If so turn it off and see if this solves the video problem. In the meantime I need to think about the other clues... -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Sun Feb 2 11:48:28 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id LAA07402 for smp-outgoing; Sun, 2 Feb 1997 11:48:28 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id LAA07387 for ; Sun, 2 Feb 1997 11:48:24 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id MAA08273; Sun, 2 Feb 1997 12:44:50 -0700 From: Terry Lambert Message-Id: <199702021944.MAA08273@phaeton.artisoft.com> Subject: Re: SMP To: davem@jenolan.rutgers.edu (David S. Miller) Date: Sun, 2 Feb 1997 12:44:50 -0700 (MST) Cc: michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org In-Reply-To: <199702021202.HAA09281@jenolan.caipgeneral> from "David S. Miller" at Feb 2, 97 07:02:25 am X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > It almost sounds like there are cases where "short holds" and "less > contention" are hard to achieve. Can you give us an example? Or > are you saying that spending time on contention minimization is not > very fruitful. > > It is hard to achieve in certain cirsumstances yes, but it is worth > putting some effort towards just not "too much" if things begin to > look a bit abismal. This is why I suggested a data-flow abstraction be the first step instead of an object abstraction. What you really want to do is lock "dangerous" areas. These are where you do shared resource manipulations within a subsystem within a context. This is an important distinction: not all objects are shared, and therefore it is not important to have all objects be locked, or even some objects may be shared in a given context and unshared in another. One good example is probably directory entry manipulation in an FS subsystem in the VFS system in a kernel. This leads to thinking of contexts in terms of "work to do", and scheduling the CPU as a shared resource against the "work to do". If you further abstract this to scheduling quantum against "work to do", you "magically" take into account kernel as well as user execution contexts. Typically, you would have the kernel stack, registers, and program counter tracked via a "kernel thread"... really, and kernel schedulable entity. Cache coherency is tracked against CPU's... it's important to note at this point the existance of MEI PPC SMP implementations without an L2 cache. User processes are a consumer of kernel schedulable entities. For instance, say you wanted minimal context switch overhead, so you design a threading system in which all system calls can be asynchronus. You provide user space with a "wait for completion" system call as the only blocking system call for an otherwise asynchronus call gate. You implement the asynchronus call gate by handing off the blocking work to a kernel schedulable entity, and let *it* block on your behalf. Quantum is now counted in terms of time in user space, and once you are given a quantum in your multithreaded application, you consume as much of the quantum as you have work to do. Another way of looking at it is "once the system *GIVES* me a quantum, by *GOD* it's *MY* quantum, and I should not be forced to give it away simply because I have made a system call... it is an unacceptable *PUNISHMENT* for making system calls". Clearly, to accommodate this, you would have to have the ability regulate the number of kernel schedulable entities which could be in user space at one time (the number of "kernel threads" a "process" has). You would also need the ability to cooperatively schedule in user space... you would implement this by giving "the execution context" (really, a kernel schedulable entity which has migrated to user space through the call gate) to a user thread that could run. Then when a thread makes a blocking call, you convert it to a non-blocking call plus a context switch (on SPARC, you also flush register windows, etc., to implement the context switch). Each kernel schedulable entity permitted in user space for a "process" competes for a process quantum as an historically implemented process. SMP scaling comes from the kernel schedulable entities in a "process" potentially having a seperate CPU resource scheduled against them. Clearly, locking occurs only in interprocessor synchronization cases, as in migration of a kernel schedulable entity from one processor to another, and/or for global pool access. A user process exists as an address space mapping, and async completion queue, and a K.S.E. -> user space quota, which can be referenced by any nimber of K.S.E.'s in the kernel, and some quota-number of K.S.E.'s in user space. So I don't lock when I enter an FS lookup, and I don't lock when I traverse a directory for read (a directory block from a read is an idempotent snapshot of the directory state), and I *do* lock when I allocate a vnode reference instance, so that I can kick the reference count. And yes, I can make the object "delete itself" on a 1->0 reference count transition, if I want to, or I can lock the vnode pool pointer in the vrele case instead, and get a minimum hold time by having it hadle the 1->0 transition. > As Terry Lambert pointed out, even if > you have per-processor pools you run into many cache pulls between > processors when you are on one cpu, allocate a resource, get scheduled > on another and then free it from there. (the TLB invalidation cases > he described are not an issue at all for us on the kernel side, brain > damaged traditional unix kernel memory designs eat this overhead, we > won't, other than vmalloc() chunks we eat no mapping setup and > destruction overhead at all and thus nothing to keep "consistant" > with IPI's since it is not changing) ??? The problem I was describing was an adjacency of allocation problem, not a mapping setup issue. Unless you are marking your allocation areas non-pageable (a really big lose, IMO), you *will* have this problem in one form or another. For instance, I get a page of memory, and I allocate two 50 byte objects out of it. I modify both of the objects, then I give the second object to another processor. The other processor modifies the second object and the first processor modifies the first object again. In theory, there will be a cache overlap, where the cache line on CPU 1 contains stale data for object two, and the cache line on CPU 2 contains stale data for object one. When either cache line is written through, the other object will be damaged, right? Not immediately, but in the case of a cache line reload. In other words, in the general case of a process context switch with a rather full ready-to-run-queue, with the resource held such that it goes out of scope in the cache. How do you resolve this? Do you allocate only in terms of an execution context, so each execution context has a kernel page pool, and then assume that if the execution context migrates, the page pool will migrate as well? How do you handle shared objects, kernel threads, and resource-based IPC? Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Sun Feb 2 12:05:56 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id MAA08311 for smp-outgoing; Sun, 2 Feb 1997 12:05:56 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA08295 for ; Sun, 2 Feb 1997 12:05:48 -0800 (PST) Received: from caipfs.rutgers.edu (root@caipfs.rutgers.edu [128.6.155.100]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id MAA11239 for ; Sun, 2 Feb 1997 12:05:45 -0800 (PST) Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id PAA07839; Sun, 2 Feb 1997 15:02:56 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id PAA18631; Sun, 2 Feb 1997 15:02:38 -0500 Date: Sun, 2 Feb 1997 15:02:38 -0500 Message-Id: <199702022002.PAA18631@jenolan.caipgeneral> From: "David S. Miller" To: terry@lambert.org CC: michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org, smpdev@roxanne.nuclecu.unam.mx In-reply-to: <199702021944.MAA08273@phaeton.artisoft.com> (message from Terry Lambert on Sun, 2 Feb 1997 12:44:50 -0700 (MST)) Subject: Re: SMP Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk From: Terry Lambert Date: Sun, 2 Feb 1997 12:44:50 -0700 (MST) For instance, I get a page of memory, and I allocate two 50 byte objects out of it. I modify both of the objects, then I give the second object to another processor. The other processor modifies the second object and the first processor modifies the first object again. In theory, there will be a cache overlap, where the cache line on CPU 1 contains stale data for object two, and the cache line on CPU 2 contains stale data for object one. When either cache line is written through, the other object will be damaged, right? Not immediately, but in the case of a cache line reload. In other words, in the general case of a process context switch with a rather full ready-to-run-queue, with the resource held such that it goes out of scope in the cache. How do you resolve this? Unless you hardware cache coherency support _really_ blows (ie. the architecture is essentially uninteresting) the most recent modification will claim ownership in that processors cache, it's called hardware cache coherency last time I checked. For example if what you are describing is: cpu1: obj1 = foo_alloc(); obj2 = foo_alloc(); dirty_up(obj1); dirty_up(obj2); /* obj2 is no longer ours. */ pass_reference(obj2, cpu2); cpu2: obj2 = get_reference_from(cpu1); dirty_up(obj2); cpu2 will see cpu1's modifications to obj2 + whatever modifications it has made on top of cpu1's. Unless you have really bad cache coherency hardware, this is guarenteed. Only exception is with store buffers, on some implementations you can have available a memory model where either: 1) stores are executed out of order 2) the store buffer is not snooped during cache transactions In such a case, most of this ugliness is hidden around memory barrier instructions (either explicitly in the code, or inside of the locking primitive implementations themselves). Or are you describing something completely different. ---------------------------------------------//// Yow! 11.26 MB/s remote host TCP bandwidth & //// 199 usec remote TCP latency over 100Mb/s //// ethernet. Beat that! //// -----------------------------------------////__________ o David S. Miller, davem@caip.rutgers.edu /_____________/ / // /_/ >< From owner-freebsd-smp Sun Feb 2 12:44:58 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id MAA11102 for smp-outgoing; Sun, 2 Feb 1997 12:44:58 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id MAA11087 for ; Sun, 2 Feb 1997 12:44:54 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id NAA08516; Sun, 2 Feb 1997 13:42:33 -0700 From: Terry Lambert Message-Id: <199702022042.NAA08516@phaeton.artisoft.com> Subject: Re: SMP To: davem@jenolan.rutgers.edu (David S. Miller) Date: Sun, 2 Feb 1997 13:42:32 -0700 (MST) Cc: terry@lambert.org, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702022002.PAA18631@jenolan.caipgeneral> from "David S. Miller" at Feb 2, 97 03:02:38 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Unless you hardware cache coherency support _really_ blows (ie. the > architecture is essentially uninteresting) the most recent > modification will claim ownership in that processors cache, it's > called hardware cache coherency last time I checked. > > For example if what you are describing is: > > cpu1: > > obj1 = foo_alloc(); > obj2 = foo_alloc(); > > dirty_up(obj1); > dirty_up(obj2); > > /* obj2 is no longer ours. */ > pass_reference(obj2, cpu2); > > cpu2: > > obj2 = get_reference_from(cpu1); > dirty_up(obj2); > > cpu2 will see cpu1's modifications to obj2 + whatever modifications it > has made on top of cpu1's. Unless you have really bad cache coherency > hardware, this is guarenteed. This is still not the problem. The problem is, what happens to obj2 when obj1 is modified and the cache line containing it is written has a piece of the old obj2, then obj2 is discarded out of cache, and then later reloaded from the memory copy which now contains corrupt data? (NOTE: I said *discarded* because CPU2 believes that the cache data is in memory because it wrote its cache line before CPU1 wrote the stale obj2 data out). > Only exception is with store buffers, on some implementations you can > have available a memory model where either: > > 1) stores are executed out of order > 2) the store buffer is not snooped during cache transactions > > In such a case, most of this ugliness is hidden around memory barrier > instructions (either explicitly in the code, or inside of the locking > primitive implementations themselves). > > Or are you describing something completely different. I'm talking case 2. <------- cache line --------> ,---------------------------, | Obj1 | Obj2 | | `---------------------------' CPU1: (* = modify) <------- cache line --------> ,---------------------------, |********| Obj2 | | --> cache line containing O1, O2 is `---------------------------' written back to main memory CPU2: (* = modify) <------- cache line --------> ,---------------------------, | Obj1 |********| | --> cache line containing O1, O2 is `---------------------------' written back to main memory CPU1 cache line now contains invalid data for Obj2. Main memory now contains stale data for Obj1. Are you claiming that CPU2 would see the write of CPU1's cache line in all cases, and invalidate it's own cache line? Isn't there a potential race? I know it's unlikely, but it exists, right? As far as I know, the write-through will not write *only* the modified bytes, right? Even if it wanted to, it would have to at *lead take it to a longword boundry, right? Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Sun Feb 2 12:52:11 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id MAA12591 for smp-outgoing; Sun, 2 Feb 1997 12:52:11 -0800 (PST) Received: from caipfs.rutgers.edu (root@caipfs.rutgers.edu [128.6.155.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA12530 for ; Sun, 2 Feb 1997 12:52:02 -0800 (PST) Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id PAA08996; Sun, 2 Feb 1997 15:51:02 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id PAA19542; Sun, 2 Feb 1997 15:50:45 -0500 Date: Sun, 2 Feb 1997 15:50:45 -0500 Message-Id: <199702022050.PAA19542@jenolan.caipgeneral> From: "David S. Miller" To: terry@lambert.org CC: terry@lambert.org, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org, smpdev@roxanne.nuclecu.unam.mx In-reply-to: <199702022042.NAA08516@phaeton.artisoft.com> (message from Terry Lambert on Sun, 2 Feb 1997 13:42:32 -0700 (MST)) Subject: Re: SMP Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk From: Terry Lambert Date: Sun, 2 Feb 1997 13:42:32 -0700 (MST) Are you claiming that CPU2 would see the write of CPU1's cache line in all cases, and invalidate it's own cache line? Isn't there a potential race? I know it's unlikely, but it exists, right? This is the classic "Writeback Cancellation Requirement", and all cache coherency hardware implementations do have to deal with it somehow. Check out section 7.10.3 of the UltraSparc Programmers Manual, in the "UltraSPARC-I External Interfaces" chapter to see how one implementation happens to deal the race very nicely. The P_Reply and S_Reply signal synchronization rules are extremely clever here. ---------------------------------------------//// Yow! 11.26 MB/s remote host TCP bandwidth & //// 199 usec remote TCP latency over 100Mb/s //// ethernet. Beat that! //// -----------------------------------------////__________ o David S. Miller, davem@caip.rutgers.edu /_____________/ / // /_/ >< From owner-freebsd-smp Sun Feb 2 13:38:20 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id NAA16297 for smp-outgoing; Sun, 2 Feb 1997 13:38:20 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id NAA16283 for ; Sun, 2 Feb 1997 13:38:17 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id OAA08696; Sun, 2 Feb 1997 14:35:27 -0700 From: Terry Lambert Message-Id: <199702022135.OAA08696@phaeton.artisoft.com> Subject: Re: SMP To: davem@jenolan.rutgers.edu (David S. Miller) Date: Sun, 2 Feb 1997 14:35:27 -0700 (MST) Cc: terry@lambert.org, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702022050.PAA19542@jenolan.caipgeneral> from "David S. Miller" at Feb 2, 97 03:50:45 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Are you claiming that CPU2 would see the write of CPU1's cache line > in all cases, and invalidate it's own cache line? Isn't there a > potential race? I know it's unlikely, but it exists, right? > > This is the classic "Writeback Cancellation Requirement", and all > cache coherency hardware implementations do have to deal with it > somehow. > > Check out section 7.10.3 of the UltraSparc Programmers Manual, in the > "UltraSPARC-I External Interfaces" chapter to see how one > implementation happens to deal the race very nicely. The P_Reply and > S_Reply signal synchronization rules are extremely clever here. Well, Sun generally does things right. I was more concerned with a number of Intel motherboards where we are seeing APIC_IO problems. They seem to fail this case. I agree that the fix for this case should be hidden in the HAL, *but* doing that would mean a severe performance hit on Intel hardware (potentially), unless the rest of the architecture were arranged so as to make it "inherently not a problem". I don't know what lists you follow, but just in the past two days, the has been a person on the FreeBSD SMP list running a motherboard with "improved cache handling" that seems to be barfing on something like this when APIC_IO is used... 8-(. Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Sun Feb 2 13:46:39 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id NAA16869 for smp-outgoing; Sun, 2 Feb 1997 13:46:39 -0800 (PST) Received: from caipfs.rutgers.edu (root@caipfs.rutgers.edu [128.6.155.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id NAA16862 for ; Sun, 2 Feb 1997 13:46:31 -0800 (PST) Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id QAA10263; Sun, 2 Feb 1997 16:44:53 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id QAA19640; Sun, 2 Feb 1997 16:44:36 -0500 Date: Sun, 2 Feb 1997 16:44:36 -0500 Message-Id: <199702022144.QAA19640@jenolan.caipgeneral> From: "David S. Miller" To: terry@lambert.org CC: terry@lambert.org, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org, smpdev@roxanne.nuclecu.unam.mx In-reply-to: <199702022135.OAA08696@phaeton.artisoft.com> (message from Terry Lambert on Sun, 2 Feb 1997 14:35:27 -0700 (MST)) Subject: Re: SMP Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk From: Terry Lambert Date: Sun, 2 Feb 1997 14:35:27 -0700 (MST) Well, Sun generally does things right. I was more concerned with a number of Intel motherboards where we are seeing APIC_IO problems. They seem to fail this case. Oh god, the APIC, programming that thing is like pulling out ones fingernails with a crowbar. They dropped those fabs on the floor big time. Intel should have sub contracted someone who knew what they were doing for that chipset. So outside of the APIC_IO cases, all else is smoothe and passes the test cases right? I mean Jesus Christ, how do you implement a read/writer lock for crying out loud with that problem? If I understand the issue properly, this means you'd have to lock the entire bus for every _access_ to any piece of the lock structure. I'm glad my primary platform isn't the Intel, but it actually a fun chip to optimize assembly for. (Note, Sun isn't the only one who got it right. SGI, HP, DEC, and just about every other hardware manufacturer whose hardware cache coherency implementation I am intimately familiar with does it right. Intel is the only fuckup case I've run into, and I had not idea it was so bad until you pointed out the problem just now.) ---------------------------------------------//// Yow! 11.26 MB/s remote host TCP bandwidth & //// 199 usec remote TCP latency over 100Mb/s //// ethernet. Beat that! //// -----------------------------------------////__________ o David S. Miller, davem@caip.rutgers.edu /_____________/ / // /_/ >< From owner-freebsd-smp Sun Feb 2 14:43:33 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id OAA18912 for smp-outgoing; Sun, 2 Feb 1997 14:43:33 -0800 (PST) Received: from himmelsborg.dna.lth.se (himmelsborg.dna.lth.se [130.235.16.11]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id OAA18902 for ; Sun, 2 Feb 1997 14:43:24 -0800 (PST) Received: (from erics@localhost) by himmelsborg.dna.lth.se (8.7.6/8.7.3/perf) id XAA05227 for <@EMIL:freebsd-smp@freebsd.org>; Sun, 2 Feb 1997 23:43:18 +0100 (MET) X-Authentication-Warning: himmelsborg.dna.lth.se: erics set sender to erics@regin.dna.lth.se using -f Received: from regin (regin [130.235.16.100]) by himmelsborg.dna.lth.se (8.7.6/8.7.3/perf) with ESMTP id XAA05213; Sun, 2 Feb 1997 23:43:10 +0100 (MET) Received: from regin by regin (SMI-8.6/) id XAA10311; Sun, 2 Feb 1997 23:43:08 +0100 Message-Id: <199702022243.XAA10311@regin> To: "David S. Miller" cc: Eric.Schenk@dna.lth.se, terry@lambert.org, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org, smpdev@roxanne.nuclecu.unam.mx From: Eric.Schenk@dna.lth.se Subject: Re: SMP In-reply-to: Your message of "Sun, 02 Feb 1997 16:44:36 EST." <199702022144.QAA19640@jenolan.caipgeneral> Date: Sun, 02 Feb 1997 23:43:06 +0100 MIME-Version: 1.0 Content-Type: Text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk "David S. Miller" writes: >I mean Jesus Christ, how do you implement a read/writer lock for >crying out loud with that problem? If I understand the issue >properly, this means you'd have to lock the entire bus for every >_access_ to any piece of the lock structure. I'm glad my primary >platform isn't the Intel, but it actually a fun chip to optimize >assembly for. ARGHH! If Intel wanted to play the multi-processor game they should have at least talked to someone who had been paying attention to the last 20 years of research in the area. Just how badly screwed are the Intel chips in this respect? My impression was that they at least where suppose to give us cache-coherency. I don't suppose that it happens to be documented when we actually get it and when we don't? Or we going to have to discover this the hard way as we progressively try to move better parallel/distibuted algorithms? -- Eric Schenk www: http://www.dna.lth.se/~erics Dept. of Comp. Sci., Lund University email: Eric.Schenk@dna.lth.se Box 118, S-221 00 LUND, Sweden fax: +46-46 13 10 21 ph: +46-46 222 96 38 From owner-freebsd-smp Sun Feb 2 14:44:08 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id OAA18967 for smp-outgoing; Sun, 2 Feb 1997 14:44:08 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id OAA18928 for ; Sun, 2 Feb 1997 14:43:59 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id PAA10876; Sun, 2 Feb 1997 15:32:16 -0700 Message-Id: <199702022232.PAA10876@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: davem@jenolan.rutgers.edu (David S. Miller), michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx Subject: Re: SMP In-reply-to: Your message of "Sun, 02 Feb 1997 14:35:27 MST." <199702022135.OAA08696@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 02 Feb 1997 15:32:16 -0700 Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk Hi, --- > Well, Sun generally does things right. I was more concerned with > a number of Intel motherboards where we are seeing APIC_IO problems. > They seem to fail this case. > ... i missed something here, could you clarify exactly what "this case" means as reguards "seeing APIC_IO problems"? --- > I don't know what lists you follow, but just in the past two days, > the has been a person on the FreeBSD SMP list running a motherboard > with "improved cache handling" that seems to be barfing on something > like this when APIC_IO is used... 8-(. "improved cache handling" refers to a cache module that actually works, ie tyan first shipped boards with a defective design, the fix for which was a new, redesigned module. It is my belief that several people are running tyan tomcat IIs with the APIC_IO option enabled. could users confirm this for us? I have a theory about why this particular board is unhappy, but I need further tests run before I can verify anything. But I don't think it has anything to do with interaction of the APIC and cache coherency. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Sun Feb 2 15:27:59 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id PAA20315 for smp-outgoing; Sun, 2 Feb 1997 15:27:59 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id PAA20308 for ; Sun, 2 Feb 1997 15:27:55 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id QAA09083; Sun, 2 Feb 1997 16:25:26 -0700 From: Terry Lambert Message-Id: <199702022325.QAA09083@phaeton.artisoft.com> Subject: Re: SMP To: smp@csn.net (Steve Passe) Date: Sun, 2 Feb 1997 16:25:26 -0700 (MST) Cc: terry@lambert.org, davem@jenolan.rutgers.edu, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702022232.PAA10876@clem.systemsix.com> from "Steve Passe" at Feb 2, 97 03:32:16 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > i missed something here, could you clarify exactly what "this case" > means as reguards "seeing APIC_IO problems"? [ ... ] > > I don't know what lists you follow, but just in the past two days, > > the has been a person on the FreeBSD SMP list running a motherboard > > with "improved cache handling" that seems to be barfing on something > > like this when APIC_IO is used... 8-(. > > "improved cache handling" refers to a cache module that actually works, > ie tyan first shipped boards with a defective design, the fix for which > was a new, redesigned module. It is my belief that several people are > running tyan tomcat IIs with the APIC_IO option enabled. could > users confirm this for us? The guy with the "new, improved cache chips" can't run APIC_IO; it was on the SMP list... both yesterday and today, in fact. He has stated that compiling without the APIC_IO runs... > I have a theory about why this particular board is unhappy, but I need > further tests run before I can verify anything. But I don't think it > has anything to do with interaction of the APIC and cache coherency. OK... your theory is probably more likely correct than mine. What do you have to say about treating the cache line coherency? Is it necessary, or is it automatic? Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Sun Feb 2 15:36:50 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id PAA20677 for smp-outgoing; Sun, 2 Feb 1997 15:36:50 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA20672 for ; Sun, 2 Feb 1997 15:36:48 -0800 (PST) Received: from caipfs.rutgers.edu (root@caipfs.rutgers.edu [128.6.155.100]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id PAA11530 for ; Sun, 2 Feb 1997 15:36:47 -0800 (PST) Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id SAA12978; Sun, 2 Feb 1997 18:34:53 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id SAA19878; Sun, 2 Feb 1997 18:34:37 -0500 Date: Sun, 2 Feb 1997 18:34:37 -0500 Message-Id: <199702022334.SAA19878@jenolan.caipgeneral> From: "David S. Miller" To: terry@lambert.org CC: smp@csn.net, terry@lambert.org, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx In-reply-to: <199702022325.QAA09083@phaeton.artisoft.com> (message from Terry Lambert on Sun, 2 Feb 1997 16:25:26 -0700 (MST)) Subject: Re: SMP Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk From: Terry Lambert Date: Sun, 2 Feb 1997 16:25:26 -0700 (MST) What do you have to say about treating the cache line coherency? Is it necessary, or is it automatic? I'm curious about Alan's theory about the situation as well. But on the whole, if a machine cannot pass the simple test you have described here, and there is no side explanation for it, chuck the machine because it is surely broken. If you start trying to code for such behavior, it will be more trouble than it's worth. Claim it broken hardware and be done with it, ahhh life is sweet again ;-) But I think this will be explained away by something else, I can't let myself believe that Intel would mess something so basic and necessary like this these days. ---------------------------------------------//// Yow! 11.26 MB/s remote host TCP bandwidth & //// 199 usec remote TCP latency over 100Mb/s //// ethernet. Beat that! //// -----------------------------------------////__________ o David S. Miller, davem@caip.rutgers.edu /_____________/ / // /_/ >< From owner-freebsd-smp Sun Feb 2 15:44:00 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id PAA20973 for smp-outgoing; Sun, 2 Feb 1997 15:44:00 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA20957 for ; Sun, 2 Feb 1997 15:43:53 -0800 (PST) Received: from snowcrash.cymru.net (snowcrash.cymru.net [163.164.160.3]) by who.cdrom.com (8.7.5/8.6.11) with ESMTP id PAA11541 for ; Sun, 2 Feb 1997 15:43:19 -0800 (PST) Received: (from alan@localhost) by snowcrash.cymru.net (8.7.1/8.7.1) id XAA01582; Sun, 2 Feb 1997 23:07:36 GMT From: Alan Cox Message-Id: <199702022307.XAA01582@snowcrash.cymru.net> Subject: Re: SMP To: terry@lambert.org (Terry Lambert) Date: Sun, 2 Feb 1997 23:07:34 +0000 (GMT) Cc: davem@jenolan.rutgers.edu, terry@lambert.org, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702022135.OAA08696@phaeton.artisoft.com> from "Terry Lambert" at Feb 2, 97 02:35:27 pm Content-Type: text Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Well, Sun generally does things right. I was more concerned with > a number of Intel motherboards where we are seeing APIC_IO problems. > They seem to fail this case. The intel boards get the hardware coherency right. After all its exactly the same for dual CPU as for CPU and PCI master. Certainly some single CPU boards get this wrong for specific cases. Just look at the matrox meteor list .. > I don't know what lists you follow, but just in the past two days, > the has been a person on the FreeBSD SMP list running a motherboard > with "improved cache handling" that seems to be barfing on something > like this when APIC_IO is used... 8-(. The APICs have a pile of little 'features'. Read the intel errata before playing. Also if you have B stepping Pentiums (except the MMX) you might as well stop on boot. Early on there were several Linux cases of 'weird crash' that came down to faulty boards/cache. Even some of them boards that worked fine single CPU. Alan From owner-freebsd-smp Sun Feb 2 16:14:05 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA22758 for smp-outgoing; Sun, 2 Feb 1997 16:14:05 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA22744 for ; Sun, 2 Feb 1997 16:13:53 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA11312; Sun, 2 Feb 1997 17:03:18 -0700 Message-Id: <199702030003.RAA11312@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: davem@jenolan.rutgers.edu, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx Subject: Re: SMP In-reply-to: Your message of "Sun, 02 Feb 1997 16:25:26 MST." <199702022325.QAA09083@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 02 Feb 1997 17:03:18 -0700 Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > > i missed something here, could you clarify exactly what "this case" > > means as reguards "seeing APIC_IO problems"? > > [ ... ] > > > > I have a theory about why this particular board is unhappy, but I need > > further tests run before I can verify anything. But I don't think it > > has anything to do with interaction of the APIC and cache coherency. > > OK... your theory is probably more likely correct than mine. > > What do you have to say about treating the cache line coherency? > Is it necessary, or is it automatic? I don't have a clue. I was under the belief that we have a MESI compliant board to deal with, but I could easily be wrong about that. I am the primary author of the FreeBSD APIC code and thus know the part pretty well. Its primary job is as a bus-oriented replacement for the 8259 ICUs. Among other features it provides IPI (InterProcessorInterrupts) facilities for communicating the need for TLB flushes, etc. between CPUs. Cache coherency is provided by the features of the P5/P6 chips and whatever MB glue their specifications require in an MP board. The Intel MP spec gives this only lip service. See the MP spec, ver 1.4, sections 3.3: External Cache SubSystem and appendix B.6: Other IPI Applications, for details. The spec can be found at: http://www.intel.com/design/pro/datashts/242016.htm Among the claims in the above mentioned spec: ------------------------------------ cut -------------------------------------- Maintaining cache coherency: When one processor accesses data cached in another processor's cache, it must not receive incorrect data. If it modifies data, all other processors that access that data also must not receive stale data. External caches must maintain coherency among themselves, and with the main memory, internal caches, and other bus master DMA devices. Cache flushing: The processor can generate special flush and write-back bus cycles that must be used by external caches in a manner that maintains cache coherency. The actual responses are implementation-specific and may vary from design to design. A program can initiate hardware cache flushing by executing a WBINVD instruction. This instruction is only guaranteed to flush the caches of the local processor. See Appendix B for system-wide flushing mechanisms. Given that cache coherency is maintained by hardware, there is no need for software to issue cache flush instructions under normal circumstances. ------------------------------------ cut -------------------------------------- --- For those of you new to the FreeBSD MP project, you can checkout: http://www.freebsd.org/~fsmp/SMP/SMP.html -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Sun Feb 2 16:20:14 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA23393 for smp-outgoing; Sun, 2 Feb 1997 16:20:14 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA23388 for ; Sun, 2 Feb 1997 16:20:09 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id RAA09824; Sun, 2 Feb 1997 17:17:39 -0700 From: Terry Lambert Message-Id: <199702030017.RAA09824@phaeton.artisoft.com> Subject: Re: SMP To: davem@jenolan.rutgers.edu (David S. Miller) Date: Sun, 2 Feb 1997 17:17:39 -0700 (MST) Cc: terry@lambert.org, smp@csn.net, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702022334.SAA19878@jenolan.caipgeneral> from "David S. Miller" at Feb 2, 97 06:34:37 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > What do you have to say about treating the cache line coherency? > Is it necessary, or is it automatic? > > I'm curious about Alan's theory about the situation as well. > > But on the whole, if a machine cannot pass the simple test you have > described here, and there is no side explanation for it, chuck the > machine because it is surely broken. > > If you start trying to code for such behavior, it will be more trouble > than it's worth. Claim it broken hardware and be done with it, ahhh > life is sweet again ;-) My initial reaction was to code to avoid triggering this class of error, and then ignoring the issue totally. The code will work whether or not the error exists. You may be right that it would be more trouble than it's worth. In the other hand, it is very tempting to make the code inherently "ruggedized" against terrible hardware. By the same argument as you are using, I could claim that most IDE hardware should be unsupported (for example). 8-). > But I think this will be explained away by something else, I can't let > myself believe that Intel would mess something so basic and necessary > like this these days. Well, I hope you are right, but given a choice between evils... 8-(. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Sun Feb 2 16:25:00 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA23802 for smp-outgoing; Sun, 2 Feb 1997 16:25:00 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA23788 for ; Sun, 2 Feb 1997 16:24:56 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id RAA09838; Sun, 2 Feb 1997 17:22:34 -0700 From: Terry Lambert Message-Id: <199702030022.RAA09838@phaeton.artisoft.com> Subject: Re: SMP To: smp@csn.net (Steve Passe) Date: Sun, 2 Feb 1997 17:22:34 -0700 (MST) Cc: terry@lambert.org, davem@jenolan.rutgers.edu, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702030003.RAA11312@clem.systemsix.com> from "Steve Passe" at Feb 2, 97 05:03:18 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > > What do you have to say about treating the cache line coherency? > > Is it necessary, or is it automatic? > > I don't have a clue. I was under the belief that we have a MESI compliant > board to deal with, but I could easily be wrong about that. You are wrong for the BeBox and the SMP PowerMac PPC603 hardware, which are only MEI (no 'S'). I don't know how Intel-centric you are, but I'd just as soon see an entry under SMP, seperate from the CPU architecture, in the build tree. > Cache flushing: > > The processor can generate special flush and write-back bus cycles > that must be used by external caches in a manner that maintains cache > coherency. The actual responses are implementation-specific and may > vary from design to design. Here's the (possible) stool sample in the flower arrangement... 8-(. If you can attribute his problems to something else, though, I will be very happy. Like I said before, it is more likely that your theory is correct (as you point out, you wrote the APIC_IO code). Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Sun Feb 2 16:30:11 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA24418 for smp-outgoing; Sun, 2 Feb 1997 16:30:11 -0800 (PST) Received: from caipfs.rutgers.edu (root@caipfs.rutgers.edu [128.6.155.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA24386 for ; Sun, 2 Feb 1997 16:30:01 -0800 (PST) Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id TAA14399; Sun, 2 Feb 1997 19:29:49 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id TAA19968; Sun, 2 Feb 1997 19:29:32 -0500 Date: Sun, 2 Feb 1997 19:29:32 -0500 Message-Id: <199702030029.TAA19968@jenolan.caipgeneral> From: "David S. Miller" To: terry@lambert.org CC: netdev@roxanne.nuclecu.unam.mx, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx In-reply-to: <199702030022.RAA09838@phaeton.artisoft.com> (message from Terry Lambert on Sun, 2 Feb 1997 17:22:34 -0700 (MST)) Subject: Re: SMP Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk From: Terry Lambert Date: Sun, 2 Feb 1997 17:22:34 -0700 (MST) [ CC: list chopped down, all removed are on the lists remaining ;-) ] You are wrong for the BeBox and the SMP PowerMac PPC603 hardware, which are only MEI (no 'S'). I don't know how Intel-centric you are, but I'd just as soon see an entry under SMP, seperate from the CPU architecture, in the build tree. I assume even without the shared cache state they do things reasonably. For example, if 1 asks for a line, and 2 has it but modified he invalidates and then 1 gets the latest copy from that invalidation. I also hope they don't invalidate in this case out to memory, if they do the bus traffic on those boxes must be unpleasant. This is a bad scheme even if 2 gets his copy stright from the others cache even though he himself is pushing straight to ram. ---------------------------------------------//// Yow! 11.26 MB/s remote host TCP bandwidth & //// 199 usec remote TCP latency over 100Mb/s //// ethernet. Beat that! //// -----------------------------------------////__________ o David S. Miller, davem@caip.rutgers.edu /_____________/ / // /_/ >< From owner-freebsd-smp Sun Feb 2 16:40:58 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA25365 for smp-outgoing; Sun, 2 Feb 1997 16:40:58 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA25360 for ; Sun, 2 Feb 1997 16:40:55 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id RAA09958; Sun, 2 Feb 1997 17:39:15 -0700 From: Terry Lambert Message-Id: <199702030039.RAA09958@phaeton.artisoft.com> Subject: Re: SMP To: davem@jenolan.rutgers.edu (David S. Miller) Date: Sun, 2 Feb 1997 17:39:15 -0700 (MST) Cc: terry@lambert.org, netdev@roxanne.nuclecu.unam.mx, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702030029.TAA19968@jenolan.caipgeneral> from "David S. Miller" at Feb 2, 97 07:29:32 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > [ CC: list chopped down, all removed are on the lists remaining ;-) ] > > You are wrong for the BeBox and the SMP PowerMac PPC603 hardware, > which are only MEI (no 'S'). I don't know how Intel-centric you > are, but I'd just as soon see an entry under SMP, seperate from > the CPU architecture, in the build tree. > > I assume even without the shared cache state they do things > reasonably. For example, if 1 asks for a line, and 2 has it but > modified he invalidates and then 1 gets the latest copy from that > invalidation. I also hope they don't invalidate in this case out to > memory, if they do the bus traffic on those boxes must be unpleasant. > This is a bad scheme even if 2 gets his copy stright from the others > cache even though he himself is pushing straight to ram. The PPC 603 SMP implementation is capable of only two processors. It operates by replacing the L2 cache with glue logic to force MEI synchronization of processor L1 cache data. I am not totally sure how this handles writes to the L2. In the Intel case, the protection against invalidation out to memory is implemented by the MMU where the "invalidate to memory" is really "invalidate to L2 cache". It is very likely that it invalidates to memory (IMO), since I have not seen a lot of complex ASIC work being described in the design (otherwise they would have, at least, a shared L2 cache). In any case, I fear Intel-centrism... Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Sun Feb 2 16:47:40 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA26275 for smp-outgoing; Sun, 2 Feb 1997 16:47:40 -0800 (PST) Received: from parkplace.cet.co.jp (parkplace.cet.co.jp [202.32.64.1]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA26266 for ; Sun, 2 Feb 1997 16:47:36 -0800 (PST) Received: from localhost (michaelh@localhost) by parkplace.cet.co.jp (8.8.5/CET-v2.1) with SMTP id AAA04791; Mon, 3 Feb 1997 00:47:06 GMT Date: Mon, 3 Feb 1997 09:47:05 +0900 (JST) From: Michael Hancock To: "David S. Miller" cc: netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org Subject: Re: SMP In-Reply-To: <199702021202.HAA09281@jenolan.caipgeneral> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Sun, 2 Feb 1997, David S. Miller wrote: > I think some key subsystems would benefit from per processor pools > (less contention) and the code can still be structured for short > holds. > [..] > On the other hand, I like the idea of: > > 1) per cpu pools for object FOO > > 2) global secondard pool for object class BAR which > FOO is a member of > > and thus the strategy becomes: > > alloc_a_FOO() > { > again: > if(FOO_free_list[smp_processor_id()]) { > /* Go like smoke */ > } else { > lock_global_pool(); > get_some_for_me(); > if(others_running_low) > feed_them_too(); > unlock_global_pool(); > goto again; > } > } > > free_a_FOO() > { > if(my_pool_only_moderately_full) { > give_it_to_me(); > } else { > feed_someone_else_who_needs_it(); > } > } > > The idea is that when you need to enter a contension situation, you > make the most of it by working towards keeping others from running > into the same situation. > > Another modification to the idea on the freeing side is to always free > this FOO to the local per-cpu queue, and if other cpus could starve > soon you give some of your oldest FOO's to them, the encourage cache > locality. > > Just some ideas. These are some of ideas I was toying about with > Larry McVoy when we were discussing with each other how N processors > could go full throttle on an N high speed interface machine with near > zero contention. I like it a lot. You can use a simple policy mechanism such as high and low watermarks for resource balancing. Terry has talked about using a global pool for resource balancing on this list, but it's nice to see in something C like. Regards, Mike Hancock From owner-freebsd-smp Sun Feb 2 16:49:50 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA26429 for smp-outgoing; Sun, 2 Feb 1997 16:49:50 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA26350 for ; Sun, 2 Feb 1997 16:48:41 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA11591 for ; Sun, 2 Feb 1997 17:47:27 -0700 Message-Id: <199702030047.RAA11591@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: freebsd-smp@FreeBSD.org Subject: Tomcat II and APIC_IO Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 02 Feb 1997 17:47:26 -0700 Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk Hi, second call, I need a showing of hands: Who has a tyan tomcat II successfully running FreeBSD SMP with APIC_IO enabled? -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Sun Feb 2 20:43:58 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id UAA07657 for smp-outgoing; Sun, 2 Feb 1997 20:43:58 -0800 (PST) Received: from smerdon.livonia.mi.us (root@pm233-12.dialip.mich.net [198.110.144.113]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id UAA07651 for ; Sun, 2 Feb 1997 20:43:53 -0800 (PST) Received: from p133 (e0.i386.smerdon.livonia.mi.us [199.33.147.37]) by smerdon.livonia.mi.us (8.7.5/8.6.9) with SMTP id XAA28684; Sun, 2 Feb 1997 23:43:36 -0500 (EST) Message-Id: <3.0.32.19970202234332.009049a0@smerdon.livonia.mi.us> X-Sender: jds@smerdon.livonia.mi.us X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Sun, 02 Feb 1997 23:43:35 -0500 To: Steve Passe From: "John D. Smerdon" Subject: Re: Tyan Tomcat II SMP video problems Cc: smp@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk At 09:53 AM 2/2/97 -0700, Steve Passe wrote: >do you have the screensaver enabled in the /etc/sysconfig file? >If so turn it off and see if this solves the video problem. The screen saver was set to "green". Setting it to NO didn't make a difference. -- John D. Smerdon; Livonia, Michigan, USA; Contents are my opinion. Home: jds@smerdon.livonia.mi.us From owner-freebsd-smp Mon Feb 3 01:37:15 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id BAA17716 for smp-outgoing; Mon, 3 Feb 1997 01:37:15 -0800 (PST) Received: from snowcrash.cymru.net (root@snowcrash.cymru.net [163.164.160.3]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id BAA17710 for ; Mon, 3 Feb 1997 01:37:08 -0800 (PST) Received: (from alan@localhost) by snowcrash.cymru.net (8.7.1/8.7.1) id JAA14489; Mon, 3 Feb 1997 09:34:43 GMT From: Alan Cox Message-Id: <199702030934.JAA14489@snowcrash.cymru.net> Subject: Re: SMP To: smp@csn.net (Steve Passe) Date: Mon, 3 Feb 1997 09:34:35 +0000 (GMT) Cc: terry@lambert.org, davem@jenolan.rutgers.edu, michaelh@cet.co.jp, netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@FreeBSD.org, smpdev@roxanne.nuclecu.unam.mx In-Reply-To: <199702030003.RAA11312@clem.systemsix.com> from "Steve Passe" at Feb 2, 97 05:03:18 pm Content-Type: text Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk > I don't have a clue. I was under the belief that we have a MESI compliant > board to deal with, but I could easily be wrong about that. You do have a MESI compliant board. > Cache coherency is provided by the features of the P5/P6 chips and > whatever MB glue their specifications require in an MP board. > > The Intel MP spec gives this only lip service. See the > MP spec, ver 1.4, sections 3.3: External Cache SubSystem and appendix > B.6: Other IPI Applications, for details. The spec can be found at: The bus state and timing diagrams for this are in the pentium pro hardware manual. Basically on a read from the other processors cache the processor with the dirty data holds the reader off and writes the modified data back sets its state to shared and then lets the other cpu continue. Alan From owner-freebsd-smp Mon Feb 3 06:43:21 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id GAA01155 for smp-outgoing; Mon, 3 Feb 1997 06:43:21 -0800 (PST) Received: from spinner.DIALix.COM (spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id GAA01148 for ; Mon, 3 Feb 1997 06:43:14 -0800 (PST) Received: from spinner.DIALix.COM (localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id WAA08729; Mon, 3 Feb 1997 22:42:09 +0800 (WST) Message-Id: <199702031442.WAA08729@spinner.DIALix.COM> X-Mailer: exmh version 2.0gamma 1/27/96 To: Kenneth Merry cc: Terje.N.Marthinussen@cc.uit.no (Terje Normann Marthinussen), smp@csn.net, smp@freebsd.org Subject: Re: Current SMP status inquiry In-reply-to: Your message of "Tue, 28 Jan 1997 12:15:26 EST." <199701281715.MAA22280@housing1.stucen.gatech.edu> Date: Mon, 03 Feb 1997 22:42:09 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk [I've been out of action for ~2 weeks and still catching up..] Kenneth Merry wrote: > Terje Normann Marthinussen wrote... [..] > > While not FPU intensive, it most certainly must have used quite a bit of > > FPU during the ~15 hours the processes ran (well, actually two runs > > of ~15 hours, a bug was discovered ;)). Didn't notice any problems at all. > > I agree, I can run FP intensive stuff, like ICO, OpenGL demos, and > all kinds of stuff without problems. I've been running X without trouble, > ever since I figured out I needed to get rid of the FP emulation stuff in > the kernel. (I accidentally left it in my config file.) The question, > though, is can you run this program successfully? > > ======================================================================== > #include > #include > void blech() { exit(3); } > main() > { > int i32; > double f; > int result = 0; > signal(SIGFPE, blech); > > f = (double) 0x7fffffff; > f = 10 * f; > i32 = (int) f; > > if (i32 != (int) f) > result |= 1; > exit(result); > } > ======================================================================== > > That program locks my machine up solid. (I got it from the perl5 > configuration script, after several attempts to compile the port hung my > machine.) It looks to me that the problem with the fpu stuff might be more a function of FPU error handling (irq13 vs. IDT trap #16) rather than just plain floating point operations... > > Ken > -- > Kenneth Merry > ken@ulc199.residence.gatech.edu > Disclaimer: I don't speak for GTRI, GT, or Elvis. Cheers, -Peter From owner-freebsd-smp Mon Feb 3 08:33:39 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id IAA06489 for smp-outgoing; Mon, 3 Feb 1997 08:33:39 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id IAA06480 for ; Mon, 3 Feb 1997 08:33:32 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id JAA15825; Mon, 3 Feb 1997 09:30:23 -0700 Message-Id: <199702031630.JAA15825@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Peter Wemm cc: smp@freebsd.org Subject: Re: Current SMP status inquiry In-reply-to: Your message of "Mon, 03 Feb 1997 22:42:09 +0800." <199702031442.WAA08729@spinner.DIALix.COM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 03 Feb 1997 09:30:23 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >> That program locks my machine up solid. (I got it from the perl5 >> configuration script, after several attempts to compile the port hung my >> machine.) > >It looks to me that the problem with the fpu stuff might be more a >function of FPU error handling (irq13 vs. IDT trap #16) rather than just >plain floating point operations... I agree, and think that someone could find it by examining what goes on in i386/isa/npx.c:mpxintr(). -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Feb 3 09:03:27 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id JAA08034 for smp-outgoing; Mon, 3 Feb 1997 09:03:27 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id JAA08022 for ; Mon, 3 Feb 1997 09:03:18 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id JAA15972; Mon, 3 Feb 1997 09:58:53 -0700 Message-Id: <199702031658.JAA15972@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: "John D. Smerdon" cc: andreas@klemm.gtn.com (Andreas Klemm), smp@freebsd.org Subject: Re: Tyan Tomcat II SMP video problems In-reply-to: Your message of "Mon, 03 Feb 1997 08:41:42 EST." <3.0.32.19970203084140.0090d830@smerdon.livonia.mi.us> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 03 Feb 1997 09:58:53 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, looks like we have the Tomcat II working with APIC_IO now: > >#define FAKE_8254_NC > >#define IRQ_LO_NC > > That did it! Thanks for your efforts! > > A message popped up during the init sequence that "APIC missing 8254 > connection" and to press return. > > I started X, compiled a kernel at "-j 8", and everything worked. > > The mptable output differences between the last attempt and this are: > [ ... ] > 170c170,172 > < Enabled INTs: 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 17, imen: 0x00fdec01 > --- > > APIC missing 8254 connection > > to continue... > > Enabled INTs: 0, 1, 3, 4, 5, 6, 7, 8, 9, 12, 17, imen: 0x00fdec04 --- I thought that other tomcat IIs were already using APIC_IO but looking thru my records there were mostly Tomcat IIIs. The only other specific reference to a Tomcat II and APIC_IO I could find was from Andreas Klemm: >I have a tyan tomcat II with 2 x 133 MHz CPU's, 512k burst cache >and 64 MB RAM. The SMP kernel runs fine, when I'm only defining > >options SMP >options NCPU=2 > >If I'n enabling the option APIC_IO, then the machine boots until >the login prompt. Shortly before it starts up the 2nd CPU, but >after that hangs. > >The keaboard is dead. An rlogin to it worked, I got an login >prompt, could login and send one command, after that the session >was frozen and I wasn't able to start another remote login. sounds VERY similar to the symptoms reported for the above board. Andreas, could you try what worked for that one: ----------------------------------- cut ------------------------------------- ok, one last thing to try, in src/sys/i386/include/smptests.h you will find 2 defines commented out: ... * #define FAKE_8254_NC #define IRQ_LO_NC */ move these to outside the comment block (ie uncomment them), build an APIC_IO kernel and see if that works. make sure to add options SMP_INVLTLB to the config file along with APIC_IO, and make NINTR == to at least 24 (or don't even bother with option NINTR). ----------------------------------- cut ------------------------------------- If this fixes Andreas's board (I suspect it will), we have a genuine rogue on our hands. The MP spec says that the 8254 timer may be left off the APIC (forcing you to do mixed mode programming with BOTH 8254s and the IO APIC). But the mp table MUST reflect this fact, which the tomcat II is FAILING to do. I will speak more on this once I get results back from Andreas or another tomcat II owner. -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Mon Feb 3 09:09:13 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id JAA08419 for smp-outgoing; Mon, 3 Feb 1997 09:09:13 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id JAA08381 for ; Mon, 3 Feb 1997 09:07:57 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA16028 for ; Mon, 3 Feb 1997 10:06:43 -0700 Message-Id: <199702031706.KAA16028@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@FreeBSD.ORG Subject: Re: Current SMP status inquiry In-reply-to: Your message of "Mon, 03 Feb 1997 09:30:23 MST." <199702031630.JAA15825@clem.systemsix.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 03 Feb 1997 10:06:43 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, > >It looks to me that the problem with the fpu stuff might be more a > >function of FPU error handling (irq13 vs. IDT trap #16) rather than just > >plain floating point operations... > > I agree, and think that someone could find it by examining what goes on > in i386/isa/npx.c:mpxintr(). A further thought, its possible that it could be APIC_IO related, could someone run this program on both an APIC_IO and non APIC_IO kernel to see if it locks both (I don't have an SMP machine anymore)? > ======================================================================== > #include > #include > void blech() { exit(3); } > main() > { > int i32; > double f; > int result = 0; > signal(SIGFPE, blech); > > f = (double) 0x7fffffff; > f = 10 * f; > i32 = (int) f; > > if (i32 != (int) f) > result |= 1; > exit(result); > } > ======================================================================== -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Feb 3 09:17:44 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id JAA08827 for smp-outgoing; Mon, 3 Feb 1997 09:17:44 -0800 (PST) Received: from star.cirrus.com (star.cirrus.com [141.131.7.10]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id JAA08822 for ; Mon, 3 Feb 1997 09:17:42 -0800 (PST) Received: from ss563.corp.cirrus.com (ss563.corp.cirrus.com [141.131.8.55]) by star.cirrus.com (8.6.12/8.6.12) with ESMTP id JAA02665 for ; Mon, 3 Feb 1997 09:17:41 -0800 Received: from elbert.colorado.cirrus.com (elbert.colorado.cirrus.com [198.90.142.11]) by ss563.corp.cirrus.com with ESMTP id JAA18135 (8.7.5/IDA-1.6 for ); Mon, 3 Feb 1997 09:17:39 -0800 (PST) Received: from longs.colorado.cirrus.com (longs.colorado.cirrus.com [198.90.139.11]) by elbert.colorado.cirrus.com with SMTP id KAA07171 (8.6.12/IDA-1.6 for ); Mon, 3 Feb 1997 10:17:50 -0700 Received: by longs.colorado.cirrus.com (4.1-Colorado/2.00) id AA14919; Mon, 3 Feb 97 10:17:50 MST Date: Mon, 3 Feb 97 10:17:50 MST From: clintw@colorado.cirrus.com (Clint Wolff) Message-Id: <9702031717.AA14919@longs.colorado.cirrus.com> To: smp@freebsd.org Subject: Re: Tyan Tomcat II SMP video problems Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > If this fixes Andreas's board (I suspect it will), we have a genuine rogue > on our hands. The MP spec says that the 8254 timer may be left off the > APIC (forcing you to do mixed mode programming with BOTH 8254s and the > IO APIC). But the mp table MUST reflect this fact, which the tomcat II > is FAILING to do. I will speak more on this once I get results back from > Andreas or another tomcat II owner. > > -- > Steve Passe | powered by > smp@csn.net | FreeBSD > I have access to three tomcat II machines, but all are running Win 95. Will the mptable program run under Win 95 or DOS? If so, I will collect data from them and post the back to you. clint From owner-freebsd-smp Mon Feb 3 11:10:21 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id LAA14161 for smp-outgoing; Mon, 3 Feb 1997 11:10:21 -0800 (PST) Received: from sendero.i-connect.net ([206.190.144.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id LAA14131; Mon, 3 Feb 1997 11:10:03 -0800 (PST) Received: (from shimon@localhost) by sendero.i-connect.net (8.8.5/8.8.4) id MAA01625; Mon, 3 Feb 1997 12:07:37 -0800 (PST) Message-ID: X-Mailer: XFMail 1.1-alpha [p0] on FreeBSD Content-Type: text/plain; charset=iso-8859-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <199702010719.SAA14427@godzilla.zeta.org.au> Date: Mon, 03 Feb 1997 11:26:23 -0800 (PST) Organization: iConnect Corp. From: Simon Shapiro To: Bruce Evans Subject: Re: CVS TAGS, whaere are you? Cc: freebsd-smp@freebsd.org, freebsd-questions@freebsd.org Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi Bruce Evans; On 01-Feb-97 you wrote: > >What are the proper tags, for CVS, to checkout the 3.0-SNAP source tree, > >the 2.2-BETA tree, and the SMP tree. I already know that the -current tree > >is ``.''. > > There aren't any for SNAPs or 2.2-BETA :-(. Tags are too expensive to > apply to SNAPs, but should be applied to BETAs. Tag RELENG_2_2 gives > the head of the 2.2 branch. You probably want that anyway unless you > are attempting to figure out which bugs are in an old version. > > Bruce THANX! About time I stop asking the same question three times :-) simon From owner-freebsd-smp Mon Feb 3 11:48:03 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id LAA16329 for smp-outgoing; Mon, 3 Feb 1997 11:48:03 -0800 (PST) Received: from gds.de (ns.gds.de [194.77.222.14]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id LAA16304 for ; Mon, 3 Feb 1997 11:47:57 -0800 (PST) Received: from pluto.gds.de (donald.plusnet.de [194.231.79.11]) by gds.de (8.8.4/8.6.12) with SMTP id UAA01807 for ; Mon, 3 Feb 1997 20:47:48 +0100 (MET) Message-Id: <199702031947.UAA01807@gds.de> Comments: Authenticated sender is From: "Richard Gresek" To: freebsd-smp@freebsd.org Date: Fri, 3 Jan 1997 20:46:12 +0000 MIME-Version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Subject: Cant spawn child process Reply-to: rg@gds.de Priority: normal X-mailer: Pegasus Mail for Win32 (v2.42a) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from Quoted-printable to 8bit by freefall.freebsd.org id LAA16305 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Great compliments to you all! The SMP-Version is running great und FAAAAAST! I have done some tests on a Gigabyta Dual P5 with 2 x 200 MHz. The machine is nearly twice as fast with SMP as with single CPU. I didnŽt expect the real results to be that near to the ideal target! I have only one problem at the moment: We are running a WWW Server with 300,000 hits a day with lot CGIs. This server is installed an a AMD 133, 64 MB RAM and FreeBSD 3.0 single CPU. On that machine all works without any problem with a load between 0.4 and 0.7. An the new machine with SMP about half of the CGIs cannot be started. The error_log reports "could not spawn child process". I have also problems with number of open files (probably this is the real cause?). I have set the values fpr number of processes per user und number of files per user each to 512, which is enough by far (the same value as on the sigle CPU machine, that runs fine.). Does someone have any idea? I have included my kernel config file at the end of this mail. Looking forward to your help Thanks in advance Richard # # GENERIC -- Generic machine with WD/AHx/NCR/BTx family disks # # For more information read the handbook part System Administration -> # Configuring the FreeBSD Kernel -> The Configuration File. # The handbook is available in /usr/share/doc/handbook or online as # latest version from the FreeBSD World Wide Web server # # # An exhaustive list of options and more detailed explanations of the # device lines is present in the ./LINT configuration file. If you are # in doubt as to the purpose or necessity of a line, check first in LINT. # # $Id: GENERIC,v 1.82 1996/12/21 02:09:04 se Exp $ machine "i386" cpu "I386_CPU" cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident GENERIC maxusers 100 options CHILD_MAX=512 options OPEN_MAX=512 options MATH_EMULATE #Support for x87 emulation options INET #InterNETworking options FFS #Berkeley Fast Filesystem options NFS #Network Filesystem options MSDOSFS #MSDOS Filesystem options "CD9660" #ISO 9660 Filesystem options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15 #Be pessimistic about Joe SCSI device options BOUNCE_BUFFERS #include support for DMA bounce buffers options UCONSOLE #Allow users to grab the console options FAILSAFE #Be conservative options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options SMP options NCPU=2 options NBUS=2 options NAPIC=1 options NINTR=24 options APIC_IO options SMP_INVLTLB options SYSVSHM options SYSVSEM options SYSVMSG config kernel root on wd0 controller isa0 controller eisa0 controller pci0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 vector fdintr disk fd0 at fdc0 drive 0 disk fd1 at fdc0 drive 1 tape ft0 at fdc0 drive 2 controller wdc0 at isa? port "IO_WD1" bio irq 14 vector wdintr disk wd0 at wdc0 drive 0 disk wd1 at wdc0 drive 1 controller wdc1 at isa? port "IO_WD2" bio irq 15 vector wdintr disk wd2 at wdc1 drive 0 disk wd3 at wdc1 drive 1 options ATAPI #Enable ATAPI support for IDE bus options ATAPI_STATIC #Don't do it as an LKM device wcd0 #IDE CD-ROM # A single entry for any of these controllers (ncr, ahb, ahc, amd) is # sufficient for any number of installed devices. controller ncr0 controller amd0 controller ahb0 controller ahc0 controller bt0 at isa? port "IO_BT0" bio irq ? vector bt_isa_intr controller uha0 at isa? port "IO_UHA0" bio irq ? drq 5 vector uhaintr controller aha0 at isa? port "IO_AHA0" bio irq ? drq 5 vector ahaintr controller aic0 at isa? port 0x340 bio irq 11 vector aicintr controller nca0 at isa? port 0x1f88 bio irq 10 vector ncaintr controller nca1 at isa? port 0x350 bio irq 5 vector ncaintr controller sea0 at isa? bio irq 5 iomem 0xc8000 iosiz 0x2000 vector seaintr controller scbus0 device sd0 device od0 #See LINT for possible `od' options. device st0 device cd0 #Only need one of these, the code dynamically grows device wt0 at isa? port 0x300 bio irq 5 drq 1 vector wtintr device mcd0 at isa? port 0x300 bio irq 10 vector mcdintr controller matcd0 at isa? port 0x230 bio device scd0 at isa? port 0x230 bio # syscons is the default console driver, resembling an SCO console device sc0 at isa? port "IO_KBD" tty irq 1 vector scintr # Enable this and PCVT_FREEBSD for pcvt vt220 compatible console driver #device vt0 at isa? port "IO_KBD" tty irq 1 vector pcrint #options PCVT_FREEBSD=210 # pcvt running on FreeBSD >= 2.0.5 #options XSERVER # include code for XFree86 #options FAT_CURSOR # start with block cursor # If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines #options PCVT_SCANSET=2 # IBM keyboards are non-std # Mandatory, don't remove device npx0 at isa? port "IO_NPX" irq 13 vector npxintr # # Laptop support (see LINT for more options) # device apm0 at isa? disable # Advanced Power Management options APM_BROKEN_STATCLOCK # Workaround some buggy APM BIOS # PCCARD (PCMCIA) support #controller crd0 #device pcic0 at crd? #device pcic1 at crd? device sio0 at isa? port "IO_COM1" tty irq 4 vector siointr device sio1 at isa? port "IO_COM2" tty irq 3 vector siointr device sio2 at isa? disable port "IO_COM3" tty irq 5 vector siointr device sio3 at isa? disable port "IO_COM4" tty irq 9 vector siointr device lpt0 at isa? port? tty irq 7 vector lptintr device lpt1 at isa? port? tty device mse0 at isa? port 0x23c tty irq 5 vector mseintr device psm0 at isa? disable port "IO_KBD" conflicts tty irq 12 vector psmintr # Order is important here due to intrusive probes, do *not* alphabetize # this list of network interfaces until the probes have been fixed. # Right now it appears that the ie0 must be probed before ep0. See # revision 1.20 of this file. device de0 device fxp0 device vx0 device ed0 at isa? port 0x280 net irq 5 iomem 0xd8000 vector edintr device ed1 at isa? port 0x300 net irq 5 iomem 0xd8000 vector edintr device ie0 at isa? port 0x360 net irq 7 iomem 0xd0000 vector ieintr device ep0 at isa? port 0x320 net irq 5 vector epintr device fe0 at isa? port 0x300 net irq ? vector feintr device ix0 at isa? port 0x300 net irq 10 iomem 0xd0000 iosiz 32768 vector ixintr device le0 at isa? port 0x300 net irq 5 iomem 0xd0000 vector le_intr device lnc0 at isa? port 0x280 net irq 10 drq 0 vector lncintr device ze0 at isa? port 0x300 net irq 5 iomem 0xd8000 vector zeintr device zp0 at isa? port 0x300 net irq 10 iomem 0xd8000 vector zpintr pseudo-device loop pseudo-device ether pseudo-device log pseudo-device sl 1 # ijppp uses tun instead of ppp device #pseudo-device ppp 1 pseudo-device tun 1 pseudo-device pty 16 pseudo-device gzip # Exec gzipped a.out's # KTRACE enables the system-call tracing facility ktrace(2). # This adds 4 KB bloat to your kernel, and slightly increases # the costs of each syscall. options KTRACE #kernel tracing +-------------------------------------------------------------------+ : Plus.Net Internet PoP fuer : Oppenheimer Landstr. 55 Frankfurt & Westerwald : 60596 Frankfurt : Tel.: +49 69 61991275 http://www.plusnet.de : Fax : +49 69 610238 +-------------------------------------------------------------------+ From owner-freebsd-smp Mon Feb 3 12:46:46 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id MAA18925 for smp-outgoing; Mon, 3 Feb 1997 12:46:46 -0800 (PST) Received: from gds.de (ns.gds.de [194.77.222.14]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA18913 for ; Mon, 3 Feb 1997 12:46:25 -0800 (PST) Received: from pluto.gds.de (donald.plusnet.de [194.231.79.11]) by gds.de (8.8.4/8.6.12) with SMTP id VAA10202 for ; Mon, 3 Feb 1997 21:46:21 +0100 (MET) Message-Id: <199702032046.VAA10202@gds.de> Comments: Authenticated sender is From: "Richard Gresek" To: freebsd-smp@freebsd.org Date: Mon, 3 Feb 1997 21:44:40 +0000 MIME-Version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Subject: 2.: Cant spawn child process Reply-to: rg@gds.de Priority: normal X-mailer: Pegasus Mail for Win32 (v2.42a) Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from Quoted-printable to 8bit by freefall.freebsd.org id MAA18921 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Great compliments to you all! !! Sorry, seems that I was not subscribed to this list, when I posted this message a few hours ago. If someone already answered, please send me a copy of your answer. ;-)) Thanks :-) The SMP-Version is running great und FAAAAAST! I have done some tests on a Gigabyta Dual P5 with 2 x 200 MHz. The machine is nearly twice as fast with SMP as with single CPU. I didnŽt expect the real results to be that near to the ideal target! I have only one problem at the moment: We are running a WWW Server with 300,000 hits a day with lot CGIs. This server is installed an a AMD 133, 64 MB RAM and FreeBSD 3.0 single CPU. On that machine all works without any problem with a load between 0.4 and 0.7. An the new machine with SMP about half of the CGIs cannot be started. The error_log reports "could not spawn child process". I have also problems with number of open files (probably this is the real cause?). I have set the values fpr number of processes per user und number of files per user each to 512, which is enough by far (the same value as on the sigle CPU machine, that runs fine.). Does someone have any idea? I have included my kernel config file at the end of this mail. Looking forward to your help Thanks in advance Richard # # GENERIC -- Generic machine with WD/AHx/NCR/BTx family disks # # For more information read the handbook part System Administration -> # Configuring the FreeBSD Kernel -> The Configuration File. # The handbook is available in /usr/share/doc/handbook or online as # latest version from the FreeBSD World Wide Web server # # # An exhaustive list of options and more detailed explanations of the # device lines is present in the ./LINT configuration file. If you are # in doubt as to the purpose or necessity of a line, check first in LINT. # # $Id: GENERIC,v 1.82 1996/12/21 02:09:04 se Exp $ machine "i386" cpu "I386_CPU" cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident GENERIC maxusers 100 options CHILD_MAX=512 options OPEN_MAX=512 options MATH_EMULATE #Support for x87 emulation options INET #InterNETworking options FFS #Berkeley Fast Filesystem options NFS #Network Filesystem options MSDOSFS #MSDOS Filesystem options "CD9660" #ISO 9660 Filesystem options PROCFS #Process filesystem options "COMPAT_43" #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15 #Be pessimistic about Joe SCSI device options BOUNCE_BUFFERS #include support for DMA bounce buffers options UCONSOLE #Allow users to grab the console options FAILSAFE #Be conservative options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options SMP options NCPU=2 options NBUS=2 options NAPIC=1 options NINTR=24 options APIC_IO options SMP_INVLTLB options SYSVSHM options SYSVSEM options SYSVMSG config kernel root on wd0 controller isa0 controller eisa0 controller pci0 controller fdc0 at isa? port "IO_FD1" bio irq 6 drq 2 vector fdintr disk fd0 at fdc0 drive 0 disk fd1 at fdc0 drive 1 tape ft0 at fdc0 drive 2 controller wdc0 at isa? port "IO_WD1" bio irq 14 vector wdintr disk wd0 at wdc0 drive 0 disk wd1 at wdc0 drive 1 controller wdc1 at isa? port "IO_WD2" bio irq 15 vector wdintr disk wd2 at wdc1 drive 0 disk wd3 at wdc1 drive 1 options ATAPI #Enable ATAPI support for IDE bus options ATAPI_STATIC #Don't do it as an LKM device wcd0 #IDE CD-ROM # A single entry for any of these controllers (ncr, ahb, ahc, amd) is # sufficient for any number of installed devices. controller ncr0 controller amd0 controller ahb0 controller ahc0 controller bt0 at isa? port "IO_BT0" bio irq ? vector bt_isa_intr controller uha0 at isa? port "IO_UHA0" bio irq ? drq 5 vector uhaintr controller aha0 at isa? port "IO_AHA0" bio irq ? drq 5 vector ahaintr controller aic0 at isa? port 0x340 bio irq 11 vector aicintr controller nca0 at isa? port 0x1f88 bio irq 10 vector ncaintr controller nca1 at isa? port 0x350 bio irq 5 vector ncaintr controller sea0 at isa? bio irq 5 iomem 0xc8000 iosiz 0x2000 vector seaintr controller scbus0 device sd0 device od0 #See LINT for possible `od' options. device st0 device cd0 #Only need one of these, the code dynamically grows device wt0 at isa? port 0x300 bio irq 5 drq 1 vector wtintr device mcd0 at isa? port 0x300 bio irq 10 vector mcdintr controller matcd0 at isa? port 0x230 bio device scd0 at isa? port 0x230 bio # syscons is the default console driver, resembling an SCO console device sc0 at isa? port "IO_KBD" tty irq 1 vector scintr # Enable this and PCVT_FREEBSD for pcvt vt220 compatible console driver #device vt0 at isa? port "IO_KBD" tty irq 1 vector pcrint #options PCVT_FREEBSD=210 # pcvt running on FreeBSD >= 2.0.5 #options XSERVER # include code for XFree86 #options FAT_CURSOR # start with block cursor # If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines #options PCVT_SCANSET=2 # IBM keyboards are non-std # Mandatory, don't remove device npx0 at isa? port "IO_NPX" irq 13 vector npxintr # # Laptop support (see LINT for more options) # device apm0 at isa? disable # Advanced Power Management options APM_BROKEN_STATCLOCK # Workaround some buggy APM BIOS # PCCARD (PCMCIA) support #controller crd0 #device pcic0 at crd? #device pcic1 at crd? device sio0 at isa? port "IO_COM1" tty irq 4 vector siointr device sio1 at isa? port "IO_COM2" tty irq 3 vector siointr device sio2 at isa? disable port "IO_COM3" tty irq 5 vector siointr device sio3 at isa? disable port "IO_COM4" tty irq 9 vector siointr device lpt0 at isa? port? tty irq 7 vector lptintr device lpt1 at isa? port? tty device mse0 at isa? port 0x23c tty irq 5 vector mseintr device psm0 at isa? disable port "IO_KBD" conflicts tty irq 12 vector psmintr # Order is important here due to intrusive probes, do *not* alphabetize # this list of network interfaces until the probes have been fixed. # Right now it appears that the ie0 must be probed before ep0. See # revision 1.20 of this file. device de0 device fxp0 device vx0 device ed0 at isa? port 0x280 net irq 5 iomem 0xd8000 vector edintr device ed1 at isa? port 0x300 net irq 5 iomem 0xd8000 vector edintr device ie0 at isa? port 0x360 net irq 7 iomem 0xd0000 vector ieintr device ep0 at isa? port 0x320 net irq 5 vector epintr device fe0 at isa? port 0x300 net irq ? vector feintr device ix0 at isa? port 0x300 net irq 10 iomem 0xd0000 iosiz 32768 vector ixintr device le0 at isa? port 0x300 net irq 5 iomem 0xd0000 vector le_intr device lnc0 at isa? port 0x280 net irq 10 drq 0 vector lncintr device ze0 at isa? port 0x300 net irq 5 iomem 0xd8000 vector zeintr device zp0 at isa? port 0x300 net irq 10 iomem 0xd8000 vector zpintr pseudo-device loop pseudo-device ether pseudo-device log pseudo-device sl 1 # ijppp uses tun instead of ppp device #pseudo-device ppp 1 pseudo-device tun 1 pseudo-device pty 16 pseudo-device gzip # Exec gzipped a.out's # KTRACE enables the system-call tracing facility ktrace(2). # This adds 4 KB bloat to your kernel, and slightly increases # the costs of each syscall. options KTRACE #kernel tracing +-------------------------------------------------------------------+ : Plus.Net Internet PoP fuer : Oppenheimer Landstr. 55 Frankfurt & Westerwald : 60596 Frankfurt : Tel.: +49 69 61991275 http://www.plusnet.de : Fax : +49 69 610238 +-------------------------------------------------------------------+ From owner-freebsd-smp Mon Feb 3 14:52:35 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id OAA25862 for smp-outgoing; Mon, 3 Feb 1997 14:52:35 -0800 (PST) Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id OAA25853 for ; Mon, 3 Feb 1997 14:52:30 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.6.12/8.6.12-s1) with UUCP id XAA12557 for smp@freebsd.org; Mon, 3 Feb 1997 23:52:19 +0100 Received: (from j@localhost) by uriah.heep.sax.de (8.8.5/8.6.9) id XAA14436; Mon, 3 Feb 1997 23:50:14 +0100 (MET) Message-ID: Date: Mon, 3 Feb 1997 23:50:09 +0100 From: j@uriah.heep.sax.de (J Wunsch) To: smp@freebsd.org Subject: My first SMP kernel... X-Mailer: Mutt 0.55-PL10 Mime-Version: 1.0 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Reply-To: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I've built my first SMP kernel today, in order to test it a little before putting the SMP source onto my CD-ROM that's going to be handed out to the participiants of the GUUG Spring Meeting by end of february. My testboard is an older ASUS PCI/EISA dual-CPU board with a Neptune chipset, running at 90 MHz still. (The second CPU was actually a 133 MHz one, but that shouldn't matter.) I've grabbed all the CTM deltas from freefall's ftp area, and checked out the most recent kernel source. Well, i didn't get this kernel to work stable, it crashed all over the place. :-( At best, it could get away to run something like `make obj', at worst, i couldn't even log in since the shell crashed immediately (sig 4, sig 11 etc.). Note that this even happened when running this kernel uni-processor, while the old (3.0-current) kernel.GENERIC ran fine on identical hardware. The SCSI controller is an AHA2742, but i didn't see SCSI errors, so it must be more a VM thingie. I tried to make the SMP options in the config file as benign as possible (just only SMP, NCPU, NBUS, and the APIC stuff, no whistles and bells), but still no go. Is this only me, am i doing something seriously wrong, or is it that i shouldn't put this source tree onto a CD-ROM at all? (Please, keep me on the Cc list.) -- cheers, J"org joerg_wunsch@uriah.heep.sax.de -- http://www.sax.de/~joerg/ -- NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) From owner-freebsd-smp Mon Feb 3 15:47:25 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id PAA28461 for smp-outgoing; Mon, 3 Feb 1997 15:47:25 -0800 (PST) Received: from po2.glue.umd.edu (root@po2.glue.umd.edu [129.2.128.45]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id PAA28455 for ; Mon, 3 Feb 1997 15:47:20 -0800 (PST) Received: from uplink.eng.umd.edu (uplink.eng.umd.edu [129.2.98.181]) by po2.glue.umd.edu (8.8.5/8.7.3) with ESMTP id SAA01929; Mon, 3 Feb 1997 18:47:15 -0500 (EST) Received: from localhost (chuckr@localhost) by uplink.eng.umd.edu (8.8.5/8.7.3) with SMTP id SAA13815; Mon, 3 Feb 1997 18:47:13 -0500 (EST) X-Authentication-Warning: uplink.eng.umd.edu: chuckr owned process doing -bs Date: Mon, 3 Feb 1997 18:47:13 -0500 (EST) From: Chuck Robey X-Sender: chuckr@uplink.eng.umd.edu To: Joerg Wunsch cc: smp@freebsd.org Subject: Re: My first SMP kernel... In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Mon, 3 Feb 1997, J Wunsch wrote: > I've built my first SMP kernel today, in order to test it a little > before putting the SMP source onto my CD-ROM that's going to be handed > out to the participiants of the GUUG Spring Meeting by end of > february. > > My testboard is an older ASUS PCI/EISA dual-CPU board with a Neptune > chipset, running at 90 MHz still. (The second CPU was actually a 133 > MHz one, but that shouldn't matter.) > > I've grabbed all the CTM deltas from freefall's ftp area, and checked > out the most recent kernel source. Well, i didn't get this kernel to > work stable, it crashed all over the place. :-( At best, it could > get away to run something like `make obj', at worst, i couldn't even > log in since the shell crashed immediately (sig 4, sig 11 etc.). Note > that this even happened when running this kernel uni-processor, while > the old (3.0-current) kernel.GENERIC ran fine on identical hardware. > > The SCSI controller is an AHA2742, but i didn't see SCSI errors, so it > must be more a VM thingie. I tried to make the SMP options in the > config file as benign as possible (just only SMP, NCPU, NBUS, and the > APIC stuff, no whistles and bells), but still no go. > > Is this only me, am i doing something seriously wrong, or is it that i > shouldn't put this source tree onto a CD-ROM at all? > > (Please, keep me on the Cc list.) Joerg, did you get the mptable program from the smp page, and what did it report about your board? http://www.freebsd.org/~fsmp/SMP/SMP.html ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Mon Feb 3 16:03:37 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA29199 for smp-outgoing; Mon, 3 Feb 1997 16:03:37 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA29151 for ; Mon, 3 Feb 1997 16:02:27 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA18147; Mon, 3 Feb 1997 17:00:36 -0700 Message-Id: <199702040000.RAA18147@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@FreeBSD.ORG cc: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) Subject: Re: My first SMP kernel... In-reply-to: Your message of "Mon, 03 Feb 1997 23:50:09 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 03 Feb 1997 17:00:36 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, > My testboard is an older ASUS PCI/EISA dual-CPU board with a Neptune > chipset, running at 90 MHz still. (The second CPU was actually a 133 > MHz one, but that shouldn't matter.) sounds a little shaky, any idea what the stepping of the original CPU is? some of the earlier P5/90s didn't do SMP very well... checkout: http://sysdoc.pair.com/pentium.html to see if this chip supports SMP --- > I've grabbed all the CTM deltas from freefall's ftp area, and checked > out the most recent kernel source. Well, i didn't get this kernel to > work stable, it crashed all over the place. :-( At best, it could > get away to run something like `make obj', at worst, i couldn't even > log in since the shell crashed immediately (sig 4, sig 11 etc.). Note > that this even happened when running this kernel uni-processor, while ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ by this you mean before doing "sysctl -w kern.smp_active=2"? --- > the old (3.0-current) kernel.GENERIC ran fine on identical hardware. I think Chuck Robey would be the one to ask as to determining whether those CTM deltas are current, I wouldn't have a clue... --- > The SCSI controller is an AHA2742, but i didn't see SCSI errors, so it > must be more a VM thingie. Peter Wemm has this controller (and I think MB) so he might have insight. --- > I tried to make the SMP options in the > config file as benign as possible (just only SMP, NCPU, NBUS, and the > APIC stuff, no whistles and bells), but still no go. did you also use options SMP_INVLTLB? this is necessary. --- > (Please, keep me on the Cc list.) > > -- > cheers, J"org -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Feb 3 16:43:16 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA02385 for smp-outgoing; Mon, 3 Feb 1997 16:43:16 -0800 (PST) Received: from po1.glue.umd.edu (root@po1.glue.umd.edu [129.2.128.44]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA02375 for ; Mon, 3 Feb 1997 16:43:12 -0800 (PST) Received: from uplink.eng.umd.edu (uplink.eng.umd.edu [129.2.98.181]) by po1.glue.umd.edu (8.8.5/8.7.3) with ESMTP id TAA07786; Mon, 3 Feb 1997 19:43:00 -0500 (EST) Received: from localhost (chuckr@localhost) by uplink.eng.umd.edu (8.8.5/8.7.3) with SMTP id TAA13723; Mon, 3 Feb 1997 19:42:58 -0500 (EST) X-Authentication-Warning: uplink.eng.umd.edu: chuckr owned process doing -bs Date: Mon, 3 Feb 1997 19:42:55 -0500 (EST) From: Chuck Robey X-Sender: chuckr@uplink.eng.umd.edu To: Steve Passe cc: smp@FreeBSD.ORG, Joerg Wunsch Subject: Re: My first SMP kernel... In-Reply-To: <199702040000.RAA18147@clem.systemsix.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk On Mon, 3 Feb 1997, Steve Passe wrote: > > I've grabbed all the CTM deltas from freefall's ftp area, and checked > > out the most recent kernel source. Well, i didn't get this kernel to > > work stable, it crashed all over the place. :-( At best, it could > > get away to run something like `make obj', at worst, i couldn't even > > log in since the shell crashed immediately (sig 4, sig 11 etc.). Note > > that this even happened when running this kernel uni-processor, while > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > by this you mean before doing "sysctl -w kern.smp_active=2"? > > --- > > the old (3.0-current) kernel.GENERIC ran fine on identical hardware. > > I think Chuck Robey would be the one to ask as to > determining whether those CTM deltas are current, I wouldn't have a clue... There hasn't been a ctm update in a couple weeks, but I haven't seen any commit messages either, so the ctm deltas available on freefall in pub/CTM/smp-cur should still represent the latest n'greatest. I built my own kernel from those sources, and it's running for me. The thing for folks to beware of is that the smp stuff doesn't reside as part of the main cvs tree, it has it's own completely separate cvs archive. In order to maintain it, you have to change your CVSROOT variable when using cvs (back and forth from pointing at /home/ncvs for the main tree, and wherever you unpack the smp cvs archive for smp). You also have to be able the mv your sys directory out of the way for building the kernel, then putting things back the way they should be before you use cvs to update your tree (else you'll trash your smp sources). I can't wait for the smp stuff to be integrated into the main tree, because doing things this way, I'm certain to bobble it, sooner or later. ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Mon Feb 3 16:51:10 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA02959 for smp-outgoing; Mon, 3 Feb 1997 16:51:10 -0800 (PST) Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA02950 for ; Mon, 3 Feb 1997 16:51:05 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.6.12/8.6.12-s1) with UUCP id BAA20285; Tue, 4 Feb 1997 01:51:03 +0100 Received: (from j@localhost) by uriah.heep.sax.de (8.8.5/8.6.9) id BAA24433; Tue, 4 Feb 1997 01:33:48 +0100 (MET) Message-ID: Date: Tue, 4 Feb 1997 01:33:47 +0100 From: j@uriah.heep.sax.de (J Wunsch) To: smp@freebsd.org Subject: Re: My first SMP kernel... References: X-Mailer: Mutt 0.55-PL10 Mime-Version: 1.0 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Reply-To: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) In-Reply-To: ; from Chuck Robey on Feb 3, 1997 18:47:13 -0500 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk As Chuck Robey wrote: > Joerg, did you get the mptable program from the smp page, and what did it > report about your board? http://www.freebsd.org/~fsmp/SMP/SMP.html Nope, didn't knew about it. Maybe i'll try to find some minute today. But i think the board's the same Peter is also using, modulo any minor revision changes, if that makes a difference. -- cheers, J"org joerg_wunsch@uriah.heep.sax.de -- http://www.sax.de/~joerg/ -- NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) From owner-freebsd-smp Mon Feb 3 16:51:05 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA02945 for smp-outgoing; Mon, 3 Feb 1997 16:51:05 -0800 (PST) Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA02933 for ; Mon, 3 Feb 1997 16:50:58 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.6.12/8.6.12-s1) with UUCP id BAA20282 for smp@FreeBSD.ORG; Tue, 4 Feb 1997 01:50:56 +0100 Received: (from j@localhost) by uriah.heep.sax.de (8.8.5/8.6.9) id BAA23817; Tue, 4 Feb 1997 01:32:04 +0100 (MET) Message-ID: Date: Tue, 4 Feb 1997 01:32:02 +0100 From: j@uriah.heep.sax.de (J Wunsch) To: smp@FreeBSD.ORG Subject: Re: My first SMP kernel... References: <199702040000.RAA18147@clem.systemsix.com> X-Mailer: Mutt 0.55-PL10 Mime-Version: 1.0 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Reply-To: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) In-Reply-To: <199702040000.RAA18147@clem.systemsix.com>; from Steve Passe on Feb 3, 1997 17:00:36 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk As Steve Passe wrote: > sounds a little shaky, any idea what the stepping of the original CPU is? It's a fairly new one. The 90 MHz chip has recently been replaced by a 100 MHz one (so we later had to find that there's no clock synthesizer on that board and that we don't have a 66 MHz oscillator lying around :). I've also verified before that both CPUs have the `SSS' signature. > > Note > > that this even happened when running this kernel uni-processor, while > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > by this you mean before doing "sysctl -w kern.smp_active=2"? No. Pulling the second CPU. Is the sysctl required to get the 2nd CPU running at all? If so, i never issued it, so i ran the machine uni-processor all the time. > did you also use options SMP_INVLTLB? this is necessary. It's been there initially, and i've took it out later, ``just in case''. Made no difference. -- cheers, J"org joerg_wunsch@uriah.heep.sax.de -- http://www.sax.de/~joerg/ -- NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) From owner-freebsd-smp Mon Feb 3 17:29:56 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id RAA05836 for smp-outgoing; Mon, 3 Feb 1997 17:29:56 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id RAA05831 for ; Mon, 3 Feb 1997 17:29:52 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id SAA18606; Mon, 3 Feb 1997 18:26:20 -0700 Message-Id: <199702040126.SAA18606@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) cc: smp@FreeBSD.ORG Subject: Re: My first SMP kernel... In-reply-to: Your message of "Tue, 04 Feb 1997 01:32:02 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 03 Feb 1997 18:26:20 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, > As Steve Passe wrote: > > > sounds a little shaky, any idea what the stepping of the original CPU is? > > It's a fairly new one. The 90 MHz chip has recently been replaced by > a 100 MHz one (so we later had to find that there's no clock > synthesizer on that board and that we don't have a 66 MHz oscillator > lying around :). I've also verified before that both CPUs have the > `SSS' signature. SSS? I don't see that in the list. what specifically are the exact S-spec #s of each? I've found that you can't trust the printout on boot of the CPU features, some versions got the bitfields messed up. > > > > Note > > > that this even happened when running this kernel uni-processor, while > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > by this you mean before doing "sysctl -w kern.smp_active=2"? > > No. Pulling the second CPU. interesting, I would have guessed this would cause boot problems, but I guess not... --- > Is the sysctl required to get the 2nd CPU running at all? If so, i > never issued it, so i ran the machine uni-processor all the time. during boot the 2nd CPU does some initialization then spins in a tight loop till the sysctl is done, so yes, you never actually were running the 2nd CPU. one possibility for testing would be to get Peter to place a known working kernel on freefall, then grab it & try, since your hardware is so similar. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Feb 3 18:15:52 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id SAA09010 for smp-outgoing; Mon, 3 Feb 1997 18:15:52 -0800 (PST) Received: from parkplace.cet.co.jp (parkplace.cet.co.jp [202.32.64.1]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id SAA09001 for ; Mon, 3 Feb 1997 18:15:49 -0800 (PST) Received: from localhost (michaelh@localhost) by parkplace.cet.co.jp (8.8.5/CET-v2.1) with SMTP id CAA15066 for ; Tue, 4 Feb 1997 02:15:45 GMT Date: Tue, 4 Feb 1997 11:15:45 +0900 (JST) From: Michael Hancock To: freebsd-smp@freebsd.org Subject: [?] TLB Flushing on Context Switch PPro vs. Pentium Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk The PPro doesn't automatically flush the TLB on context switch like the Pentium did. What else does the PPro do differently? Mike Hancock From owner-freebsd-smp Tue Feb 4 00:50:37 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id AAA02317 for smp-outgoing; Tue, 4 Feb 1997 00:50:37 -0800 (PST) Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id AAA02306 for ; Tue, 4 Feb 1997 00:50:33 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.6.12/8.6.12-s1) with UUCP id JAA27630 for smp@FreeBSD.ORG; Tue, 4 Feb 1997 09:50:31 +0100 Received: (from j@localhost) by uriah.heep.sax.de (8.8.5/8.6.9) id JAA27459; Tue, 4 Feb 1997 09:27:07 +0100 (MET) Message-ID: Date: Tue, 4 Feb 1997 09:27:07 +0100 From: j@uriah.heep.sax.de (J Wunsch) To: smp@FreeBSD.ORG Subject: Re: My first SMP kernel... References: <199702040126.SAA18606@clem.systemsix.com> X-Mailer: Mutt 0.55-PL10 Mime-Version: 1.0 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Reply-To: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) In-Reply-To: <199702040126.SAA18606@clem.systemsix.com>; from Steve Passe on Feb 3, 1997 18:26:20 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk As Steve Passe wrote: > > lying around :). I've also verified before that both CPUs have the > > `SSS' signature. > > SSS? I don't see that in the list. The numbers on the bottom lid end up in SSS. > what specifically are the exact > S-spec #s of each? I have to look, they are at work (but i'm at home still). > one possibility for testing would be to get Peter to place a known working > kernel on freefall, then grab it & try, since your hardware is so similar. Well, my question was rather whether i should basically mistrust that particular source tree (and re-roll that CD-ROM for the GUUG meeting again), or whether it's just a personal problem with my setup. In the latter case, it's not really important, i've only dug up that old Neptune board for a simple test. (It's our scratch machine at work, but there's usually only a single CPU in it, i have to ``borrow'' another CPU from another machine for testing.) What surprised me was that this SMP kernel crashed all over the place even on the board equipped with a single CPU only, while the kernel.GENERIC worked well with the very same hardware. I probably (unfortunately) won't find the time to track this into every detail right now. But maybe i'm re-rolling the bits for that CD-ROM anyway, so i will at least stuff the comment about the sysctl into my README.SMP there. -- cheers, J"org joerg_wunsch@uriah.heep.sax.de -- http://www.sax.de/~joerg/ -- NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) From owner-freebsd-smp Tue Feb 4 01:51:25 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id BAA06161 for smp-outgoing; Tue, 4 Feb 1997 01:51:25 -0800 (PST) Received: from sendero.i-connect.net ([206.190.144.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id BAA06150 for ; Tue, 4 Feb 1997 01:51:04 -0800 (PST) Received: (from shimon@localhost) by sendero.i-connect.net (8.8.5/8.8.4) id CAA26158 for freebsd-smp@freebsd.org; Tue, 4 Feb 1997 02:50:15 -0800 (PST) Message-ID: X-Mailer: XFMail 1.1-alpha [p0] on FreeBSD Content-Type: text/plain; charset=iso-8859-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 Date: Tue, 04 Feb 1997 02:37:00 -0800 (PST) Organization: iConnect Corp. From: Simon Shapiro To: freebsd-smp@freebsd.org Subject: CTM Problem Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk As Suggested, I am trying to create the source tree from ctm. Here is where things break: $ctm -v -u -F /Archives/FreeBSD/CTM/smp-cur/smp-cur.0050.gz Working on . failed date validation /Archives/FreeBSD/CTM/smp-cur/smp-cur.0050.gz Fatal error: Assert failed. Exit(96) After that, I tried skipping 50 (it appears empty) but 51 gave me the same results. What am I doing wrong? Simon From owner-freebsd-smp Tue Feb 4 03:48:15 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id DAA11967 for smp-outgoing; Tue, 4 Feb 1997 03:48:15 -0800 (PST) Received: from dyson.iquest.net (dyson.iquest.net [198.70.144.127]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id DAA11962 for ; Tue, 4 Feb 1997 03:48:10 -0800 (PST) Received: (from root@localhost) by dyson.iquest.net (8.8.4/8.6.9) id GAA05042; Tue, 4 Feb 1997 06:47:52 -0500 (EST) From: "John S. Dyson" Message-Id: <199702041147.GAA05042@dyson.iquest.net> Subject: Re: [?] TLB Flushing on Context Switch PPro vs. Pentium To: michaelh@cet.co.jp (Michael Hancock) Date: Tue, 4 Feb 1997 06:47:52 -0500 (EST) Cc: freebsd-smp@freebsd.org In-Reply-To: from "Michael Hancock" at Feb 4, 97 11:15:45 am X-Mailer: ELM [version 2.4 PL24 ME8] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > The PPro doesn't automatically flush the TLB on context switch like the > Pentium did. What else does the PPro do differently? > On the non-SMP kernel, we specifically set the PG_G flag that keeps the kernel pages in the TLB on PPro. I have also been playing with the idea of using 4MB pages (which the P5 also unofficially supports.) John From owner-freebsd-smp Tue Feb 4 05:28:59 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id FAA15679 for smp-outgoing; Tue, 4 Feb 1997 05:28:59 -0800 (PST) Received: from spinner.DIALix.COM (spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA15674 for ; Tue, 4 Feb 1997 05:28:51 -0800 (PST) Received: from spinner.DIALix.COM (localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id VAA15354; Tue, 4 Feb 1997 21:27:43 +0800 (WST) Message-Id: <199702041327.VAA15354@spinner.DIALix.COM> X-Mailer: exmh version 2.0gamma 1/27/96 To: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) cc: smp@FreeBSD.ORG Subject: Re: My first SMP kernel... In-reply-to: Your message of "Tue, 04 Feb 1997 09:27:07 +0100." Date: Tue, 04 Feb 1997 21:27:42 +0800 From: Peter Wemm Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk J Wunsch wrote: > As Steve Passe wrote: > > > > lying around :). I've also verified before that both CPUs have the > > > `SSS' signature. > > > > SSS? I don't see that in the list. > > The numbers on the bottom lid end up in SSS. SSS chips are theoretically the "best you can get".. ie: they don't need voltage twiddles etc to get them to work. > > what specifically are the exact > > S-spec #s of each? > > I have to look, they are at work (but i'm at home still). What do you get on the dmesg output when booting a non-smp kernel? CPU: Pentium (90.00-MHz 586-class CPU) Origin = "GenuineIntel" Id = 0x521 Stepping=1 Features=0x7bf> [...] npx0 on motherboard npx0: INT 16 interface npx0: Pentium floating point divide (fdiv) flaw present! This comes from a very early batch of P5-90's. > > one possibility for testing would be to get Peter to place a known working > > kernel on freefall, then grab it & try, since your hardware is so similar. > > Well, my question was rather whether i should basically mistrust that > particular source tree (and re-roll that CD-ROM for the GUUG meeting > again), or whether it's just a personal problem with my setup. In the > latter case, it's not really important, i've only dug up that old > Neptune board for a simple test. (It's our scratch machine at work, > but there's usually only a single CPU in it, i have to ``borrow'' > another CPU from another machine for testing.) I seem to recall dire warnings about mixing different stepping/S number chips from the early runs.. I believe the apic timing etc was tweaked a bit early on, but they now seem to have it sorted out.. But if one of the chips is old, you might be in for fun. > What surprised me was that this SMP kernel crashed all over the place > even on the board equipped with a single CPU only, while the > kernel.GENERIC worked well with the very same hardware. The SMP kernel is very out of date with respect to -current. I believe it's missing some VM fixes from John Dyson. Now that I've nearly finished moving house (argh, what a nightmare!) I must revisit this. I don't think there are any fundamental differences, except for the MAXLOGNAME size bump (it's 12 in SMP, but 16 in -current with padding to make the structures compatable). Hmm, I think I'd better import a new version before John strikes with the Lite2 merge in -current.. :-] Cheers, -Peter From owner-freebsd-smp Tue Feb 4 05:39:02 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id FAA16071 for smp-outgoing; Tue, 4 Feb 1997 05:39:02 -0800 (PST) Received: from spinner.DIALix.COM (spinner.DIALix.COM [192.203.228.67]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA16060 for ; Tue, 4 Feb 1997 05:38:51 -0800 (PST) Received: from spinner.DIALix.COM (localhost.DIALix.oz.au [127.0.0.1]) by spinner.DIALix.COM (8.8.4/8.8.4) with ESMTP id VAA15405; Tue, 4 Feb 1997 21:37:21 +0800 (WST) Message-Id: <199702041337.VAA15405@spinner.DIALix.COM> X-Mailer: exmh version 2.0gamma 1/27/96 To: "John S. Dyson" cc: michaelh@cet.co.jp (Michael Hancock), freebsd-smp@freebsd.org Subject: Re: [?] TLB Flushing on Context Switch PPro vs. Pentium In-reply-to: Your message of "Tue, 04 Feb 1997 06:47:52 EST." <199702041147.GAA05042@dyson.iquest.net> Date: Tue, 04 Feb 1997 21:37:20 +0800 From: Peter Wemm Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk "John S. Dyson" wrote: > > The PPro doesn't automatically flush the TLB on context switch like the > > Pentium did. What else does the PPro do differently? > > > On the non-SMP kernel, we specifically set the PG_G flag that > keeps the kernel pages in the TLB on PPro. I have also been > playing with the idea of using 4MB pages (which the P5 also > unofficially supports.) > > John The PPro also flushes something in it's pipelines and/or prefetch during context switch (ie: reload %cr3) while the P5 does not. We got bitten with this with a bug in the startup code when we first tried it on a P6. The code basically was: movl $_PTD,%eax movl %eax,%cr3 jmp new_address The problem was that the 2nd cpu was running with a 4MB mapping at address 0 with Virtual == Physical at that stage. When going into full virtual mode, it loaded up the PTD pointer and jumped into the real location of the kernel.. But doing so also removed the P=V at 0, so the code disappeared right underneath the flow of execution. The P5 didn't care, it obviously had enough in it's pipeline to keep going, but the P6 spat the dummy (as should happen on all cpu's, or bugs like the above code go unnoticed). Cheers, -Peter From owner-freebsd-smp Tue Feb 4 05:52:12 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id FAA16484 for smp-outgoing; Tue, 4 Feb 1997 05:52:12 -0800 (PST) Received: from dyson.iquest.net (dyson.iquest.net [198.70.144.127]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id FAA16478 for ; Tue, 4 Feb 1997 05:52:09 -0800 (PST) Received: (from root@localhost) by dyson.iquest.net (8.8.4/8.6.9) id IAA05279; Tue, 4 Feb 1997 08:51:14 -0500 (EST) From: "John S. Dyson" Message-Id: <199702041351.IAA05279@dyson.iquest.net> Subject: Re: My first SMP kernel... To: peter@spinner.dialix.com (Peter Wemm) Date: Tue, 4 Feb 1997 08:51:14 -0500 (EST) Cc: joerg_wunsch@uriah.heep.sax.de, smp@FreeBSD.ORG In-Reply-To: <199702041327.VAA15354@spinner.DIALix.COM> from "Peter Wemm" at Feb 4, 97 09:27:42 pm X-Mailer: ELM [version 2.4 PL24 ME8] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > > The SMP kernel is very out of date with respect to -current. I believe > it's missing some VM fixes from John Dyson. Now that I've nearly finished > moving house (argh, what a nightmare!) I must revisit this. I don't think > there are any fundamental differences, except for the MAXLOGNAME size bump > (it's 12 in SMP, but 16 in -current with padding to make the structures > compatable). Hmm, I think I'd better import a new version before John > strikes with the Lite2 merge in -current.. :-] > I have only to fix procfs, and will be posting a message about a patch file that will patch -current up to the Lite/2 kernel (this week.) Won't commit them without at least a day warning. John From owner-freebsd-smp Tue Feb 4 08:13:55 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id IAA23835 for smp-outgoing; Tue, 4 Feb 1997 08:13:55 -0800 (PST) Received: from deepo.prosa.dk ([193.89.187.27]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id IAA23822; Tue, 4 Feb 1997 08:13:47 -0800 (PST) Received: (from regnauld@localhost) by deepo.prosa.dk (8.8.5/8.8.4/prosa-1.1) id RAA11142; Tue, 4 Feb 1997 17:14:32 +0100 (CET) Message-ID: Date: Tue, 4 Feb 1997 17:14:32 +0100 From: regnauld@deepo.prosa.dk (Philippe Regnauld) To: freebsd-smp@freebsd.org, freebsd-hardware@freebsd.org Subject: Finding Gigabyte 586-DX MB in Europe ? X-Mailer: Mutt 0.58 Mime-Version: 1.0 X-Operating-System: FreeBSD 2.2-BETA_A i386 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk ... Where might I find one of those critters, preferably in the Northern half of Europe ? SMP is itching :-) -- -- Phil -[ Philippe Regnauld / Systems Administrator / regnauld@.prosa.dk ]- -[ Location.: +55.4N +11.3E PGP Key: finger regnauld@deepo.prosa.dk ]- From owner-freebsd-smp Tue Feb 4 09:10:53 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id JAA28754 for smp-outgoing; Tue, 4 Feb 1997 09:10:53 -0800 (PST) Received: from ravenock.cybercity.dk (ravenock.cybercity.dk [194.16.57.32]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id JAA28726; Tue, 4 Feb 1997 09:10:44 -0800 (PST) Received: (from sos@localhost) by ravenock.cybercity.dk (8.8.5/8.7.3) id SAA25705; Tue, 4 Feb 1997 18:12:15 +0100 (MET) From: Søren Schmidt Message-Id: <199702041712.SAA25705@ravenock.cybercity.dk> Subject: Re: Finding Gigabyte 586-DX MB in Europe ? In-Reply-To: from Philippe Regnauld at "Feb 4, 97 05:14:32 pm" To: regnauld@deepo.prosa.dk (Philippe Regnauld) Date: Tue, 4 Feb 1997 18:12:14 +0100 (MET) Cc: freebsd-smp@FreeBSD.ORG, freebsd-hardware@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL30 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk In reply to Philippe Regnauld who wrote: > ... Where might I find one of those critters, preferably in the Northern > half of Europe ? SMP is itching :-) Try: Frish Data Munkevej 3 3500 Værløse 4447 1281 They carry at least some of the Gigabyte MB's For more info on danish (enduser) prices, look at: http://home5.inet.tele.dk/wheeler/priser.htm Its a guy that collects prices from many dealers, and compares them on given brand etc etc... -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Søren Schmidt (sos@FreeBSD.org) FreeBSD Core Team Even more code to hack -- will it ever end .. From owner-freebsd-smp Tue Feb 4 12:50:34 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id MAA11117 for smp-outgoing; Tue, 4 Feb 1997 12:50:34 -0800 (PST) Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id MAA11112 for ; Tue, 4 Feb 1997 12:50:31 -0800 (PST) Received: from current1.whistle.com (current1.whistle.com [207.76.205.22]) by alpo.whistle.com (8.8.5/8.8.4) with SMTP id MAA09915; Tue, 4 Feb 1997 12:43:43 -0800 (PST) Message-ID: <32F79F16.41C67EA6@whistle.com> Date: Tue, 04 Feb 1997 12:41:58 -0800 From: Julian Elischer Organization: Whistle Communications X-Mailer: Mozilla 3.0Gold (X11; I; FreeBSD 2.2-CURRENT i386) MIME-Version: 1.0 To: "John S. Dyson" CC: Michael Hancock , freebsd-smp@freebsd.org Subject: Re: [?] TLB Flushing on Context Switch PPro vs. Pentium References: <199702041147.GAA05042@dyson.iquest.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk John S. Dyson wrote: > > > The PPro doesn't automatically flush the TLB on context switch like the > > Pentium did. What else does the PPro do differently? > > > On the non-SMP kernel, we specifically set the PG_G flag that > keeps the kernel pages in the TLB on PPro. I have also been > playing with the idea of using 4MB pages (which the P5 also > unofficially supports.) I believe linux now does this. > > John From owner-freebsd-smp Tue Feb 4 14:33:25 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id OAA18863 for smp-outgoing; Tue, 4 Feb 1997 14:33:25 -0800 (PST) Received: from sax.sax.de (sax.sax.de [193.175.26.33]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id OAA18848 for ; Tue, 4 Feb 1997 14:33:19 -0800 (PST) Received: (from uucp@localhost) by sax.sax.de (8.6.12/8.6.12-s1) with UUCP id XAA21577 for smp@FreeBSD.ORG; Tue, 4 Feb 1997 23:33:17 +0100 Received: (from j@localhost) by uriah.heep.sax.de (8.8.5/8.6.9) id WAA29131; Tue, 4 Feb 1997 22:36:50 +0100 (MET) Message-ID: Date: Tue, 4 Feb 1997 22:36:50 +0100 From: j@uriah.heep.sax.de (J Wunsch) To: smp@FreeBSD.ORG Subject: Re: My first SMP kernel... References: <199702041327.VAA15354@spinner.DIALix.COM> X-Mailer: Mutt 0.55-PL10 Mime-Version: 1.0 X-Phone: +49-351-2012 669 X-PGP-Fingerprint: DC 47 E6 E4 FF A6 E9 8F 93 21 E0 7D F9 12 D6 4E Reply-To: joerg_wunsch@uriah.heep.sax.de (Joerg Wunsch) In-Reply-To: <199702041327.VAA15354@spinner.DIALix.COM>; from Peter Wemm on Feb 4, 1997 21:27:42 +0800 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk As Peter Wemm wrote: > > The numbers on the bottom lid end up in SSS. > > SSS chips are theoretically the "best you can get".. ie: they don't need > voltage twiddles etc to get them to work. Yes, i also thought so. > What do you get on the dmesg output when booting a non-smp kernel? I have to look, but as i wrote earlier, it will probably take some time until this board will boot a freebsd disk for the next time. > npx0: INT 16 interface > npx0: Pentium floating point divide (fdiv) flaw present! No, i haven't seen the latter. I didn't even know this message exists. :) Last time i've been working with such a flawed P90, this was at my previous employer under FreeBSD 1.1.5.1 (which is still working there). > I seem to recall dire warnings about mixing different stepping/S number > chips from the early runs.. I believe the apic timing etc was tweaked a > bit early on, but they now seem to have it sorted out.. But if one of the > chips is old, you might be in for fun. No, both my CPUs were rather recent. And, i ran into those troubles even with only one CPU (but the SMP kernel). The machine suffered from some SCSI bus mistermination earlier. Maybe it was just this, and some broken byte sneaked into some C or object file. I think i'll redo the experiment some day again. -- cheers, J"org joerg_wunsch@uriah.heep.sax.de -- http://www.sax.de/~joerg/ -- NIC: JW11-RIPE Never trust an operating system you don't have sources for. ;-) From owner-freebsd-smp Tue Feb 4 17:05:54 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id RAA04093 for smp-outgoing; Tue, 4 Feb 1997 17:05:54 -0800 (PST) Received: from rocky.mt.sri.com (rocky.mt.sri.com [206.127.76.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id RAA04082 for ; Tue, 4 Feb 1997 17:05:49 -0800 (PST) Received: (from nate@localhost) by rocky.mt.sri.com (8.7.5/8.7.3) id SAA09082; Tue, 4 Feb 1997 18:05:24 -0700 (MST) Date: Tue, 4 Feb 1997 18:05:24 -0700 (MST) Message-Id: <199702050105.SAA09082@rocky.mt.sri.com> From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Julian Elischer Cc: "John S. Dyson" , Michael Hancock , freebsd-smp@freebsd.org Subject: Re: [?] TLB Flushing on Context Switch PPro vs. Pentium In-Reply-To: <32F79F16.41C67EA6@whistle.com> References: <199702041147.GAA05042@dyson.iquest.net> <32F79F16.41C67EA6@whistle.com> Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > > The PPro doesn't automatically flush the TLB on context switch like the > > > Pentium did. What else does the PPro do differently? > > > > > On the non-SMP kernel, we specifically set the PG_G flag that > > keeps the kernel pages in the TLB on PPro. I have also been > > playing with the idea of using 4MB pages (which the P5 also > > unofficially supports.) > > I believe linux now does this. I know Linus toyed with this, but ended up going back due to bugs in many Pentium chips. (My Pentium will not work with 4MB pages, and it's a P100 w/out the FDIV bug.) Nate From owner-freebsd-smp Tue Feb 4 17:27:22 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id RAA07731 for smp-outgoing; Tue, 4 Feb 1997 17:27:22 -0800 (PST) Received: from caipfs.rutgers.edu (root@caipfs.rutgers.edu [128.6.155.100]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id RAA07722 for ; Tue, 4 Feb 1997 17:27:19 -0800 (PST) Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id UAA05067; Tue, 4 Feb 1997 20:24:28 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id UAA03391; Tue, 4 Feb 1997 20:24:12 -0500 Date: Tue, 4 Feb 1997 20:24:12 -0500 Message-Id: <199702050124.UAA03391@jenolan.caipgeneral> From: "David S. Miller" To: nate@mt.sri.com CC: julian@whistle.com, toor@dyson.iquest.net, michaelh@cet.co.jp, freebsd-smp@freebsd.org In-reply-to: <199702050105.SAA09082@rocky.mt.sri.com> (message from Nate Williams on Tue, 4 Feb 1997 18:05:24 -0700 (MST)) Subject: Re: [?] TLB Flushing on Context Switch PPro vs. Pentium Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Date: Tue, 4 Feb 1997 18:05:24 -0700 (MST) From: Nate Williams > I believe linux now does this. I know Linus toyed with this, but ended up going back due to bugs in many Pentium chips. (My Pentium will not work with 4MB pages, and it's a P100 w/out the FDIV bug.) Ifdef'd out by default in 2.0.x, if enabled in that code it can then be dynamically disabled via the mem=nopentium boot time command line option. In 2.1.x it is completely enabled. From owner-freebsd-smp Tue Feb 4 22:34:54 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id WAA07676 for smp-outgoing; Tue, 4 Feb 1997 22:34:54 -0800 (PST) Received: from pdx1.world.net (pdx1.world.net [192.243.32.18]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id WAA07667 for ; Tue, 4 Feb 1997 22:34:51 -0800 (PST) Received: from suburbia.net (suburbia.net [203.4.184.1]) by pdx1.world.net (8.7.5/8.7.3) with SMTP id WAA11021 for ; Tue, 4 Feb 1997 22:36:02 -0800 (PST) Received: (qmail 28573 invoked by uid 110); 5 Feb 1997 06:33:51 -0000 MBOX-Line: From owner-netdev@roxanne.nuclecu.unam.mx Wed Feb 05 04:34:48 1997 remote from suburbia.net Delivered-To: proff@suburbia.net Received: (qmail 4637 invoked from network); 5 Feb 1997 04:17:35 -0000 Received: from roxanne.nuclecu.unam.mx (132.248.29.2) by suburbia.net with SMTP; 5 Feb 1997 04:17:35 -0000 Received: (from root@localhost) by roxanne.nuclecu.unam.mx (8.6.12/8.6.11) id WAA23063 for netdev-outgoing; Tue, 4 Feb 1997 22:08:17 -0600 Received: from caipfs.rutgers.edu (caipfs.rutgers.edu [128.6.155.100]) by roxanne.nuclecu.unam.mx (8.6.12/8.6.11) with ESMTP id WAA23058; Tue, 4 Feb 1997 22:08:04 -0600 Received: from jenolan.caipgeneral (jenolan.rutgers.edu [128.6.111.5]) by caipfs.rutgers.edu (8.7.6/8.7.3) with SMTP id XAA11274; Tue, 4 Feb 1997 23:05:25 -0500 (EST) Received: by jenolan.caipgeneral (SMI-8.6/SMI-SVR4) id XAA03584; Tue, 4 Feb 1997 23:05:08 -0500 Date: Tue, 4 Feb 1997 23:05:08 -0500 Message-Id: <199702050405.XAA03584@jenolan.caipgeneral> From: "David S. Miller" To: netdev@roxanne.nuclecu.unam.mx CC: smpdev@roxanne.nuclecu.unam.mx Subject: paper I stumbled across Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk vger.rutgers.edu:/pub/linux/SMP/papers/netlocking.ps.gz It discusses the parallelization of the xkernel network protocol code for shared memory multiprocessors. It includes analysis of various simulations that the authors ran over their implementation using various traces of tcp and udp connection sets. They also explore strategies, payoff, and the overhead of the locking itself on the operations that need to be performed. From owner-freebsd-smp Wed Feb 5 11:11:47 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id LAA02781 for smp-outgoing; Wed, 5 Feb 1997 11:11:47 -0800 (PST) Received: from kremvax.demos.su (kremvax.demos.su [194.87.0.20]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id LAA02761 for ; Wed, 5 Feb 1997 11:11:33 -0800 (PST) Received: by kremvax.demos.su (8.6.13/D) from 0@sinbin.demos.su [194.87.2.95] with ESMTP id WAA25207; Wed, 5 Feb 1997 22:10:50 +0300 Received: by sinbin.demos.su id WAA09819; (8.6.12/D) Wed, 5 Feb 1997 22:10:44 +0300 From: bag@sinbin.demos.su (Alex G. Bulushev) Message-Id: <199702051910.WAA09819@sinbin.demos.su> Subject: Re: troubles with smp kernel To: smp@csn.net (Steve Passe) Date: Wed, 5 Feb 1997 22:10:44 +0300 (MSK) Cc: freebsd-smp@freebsd.org, mishania@demos.su In-Reply-To: <199701311835.LAA24610@clem.systemsix.com> from "Steve Passe" at Jan 31, 97 11:35:50 am X-Mailer: ELM [version 2.4 PL24 ME7a] Content-Type: text Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Perhaps its the 2nd 3940 itself? After running without rebooting for a > day or so I would suggest that you swap the 2 3940s, ie replace the working > one with the currently removed one. Then run with it alone (ie leave the > known good one out) and see if this 2nd card also runs by itself withoout > problem. > We try to run solaris x86 2.5 on this PC and it works without any problems, after booting fbsd with kernel/smp it reboots. Average stable time about 1h, maximum time before reboots about 6h. Now we use single 3940 and there is no sharing irq's. We change our ram, and there is no parity error, we change one CPU and now they have the same step = 6 ... mptable output: =============================================================================== MPTable, version 2.0.6 ------------------------------------------------------------------------------- MP Floating Pointer Structure: location: BIOS physical address: 0x000f60b0 signature: '_MP_' length: 16 bytes version: 1.4 checksum: 0x8b mode: Virtual Wire ------------------------------------------------------------------------------- MP Config Table Header: physical address: 0x000f5caa signature: 'PCMP' base table length: 252 version: 1.4 checksum: 0xb2 OEM ID: 'OEM00000' Product ID: 'PROD00000000' OEM table pointer: 0x00000000 OEM table size: 0 entry count: 23 local APIC address: 0xfee00000 extended table length: 0 extended table checksum: 0 ------------------------------------------------------------------------------- MP Config Base Table Entries: -- Processors: APIC ID Version State Family Model Step Flags 1 0x11 BSP, usable 6 1 6 0xfbff 0 0x11 AP, usable 6 1 6 0xfbff -- Bus: Bus ID Type 0 PCI 1 PCI 2 ISA -- I/O APICs: APIC ID Version State Address 2 0x11 usable 0xfec00000 -- I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT conforms conforms 2 0 2 0 INT conforms conforms 2 1 2 1 INT conforms conforms 2 0 2 2 INT conforms conforms 2 3 2 3 INT conforms conforms 2 4 2 4 INT conforms conforms 2 5 2 5 INT conforms conforms 2 6 2 6 INT conforms conforms 2 7 2 7 INT conforms conforms 2 8 2 8 INT conforms conforms 2 9 2 9 INT conforms conforms 2 14 2 14 INT conforms conforms 2 15 2 15 INT active-lo level 1 4:A 2 19 INT active-lo level 1 5:A 2 16 INT active-lo level 0 10:A 2 18 -- Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID INT# ExtINT active-hi edge 2 0 255 0 NMI active-hi edge 2 0 255 1 ------------------------------------------------------------------------------- # SMP kernel config file options: options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O options NCPU=2 # number of CPUs options NBUS=3 # number of busses options NAPIC=1 # number of IO APICs options NINTR=24 # number of INTs options SMP_INVLTLB # #options SMP_PRIVPAGES # BROKEN, DO NOT use! #options SMP_AUTOSTART # BROKEN, DO NOT use! #options SERIAL_DEBUG # com port debug output =============================================================================== From owner-freebsd-smp Wed Feb 5 11:27:20 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id LAA04077 for smp-outgoing; Wed, 5 Feb 1997 11:27:20 -0800 (PST) Received: from kremvax.demos.su (kremvax.demos.su [194.87.0.20]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id LAA04018 for ; Wed, 5 Feb 1997 11:26:37 -0800 (PST) Received: by kremvax.demos.su (8.6.13/D) from 0@sinbin.demos.su [194.87.2.95] with ESMTP id WAA29087; Wed, 5 Feb 1997 22:25:57 +0300 Received: by sinbin.demos.su id WAA11966; (8.6.12/D) Wed, 5 Feb 1997 22:25:20 +0300 From: bag@sinbin.demos.su (Alex G. Bulushev) Message-Id: <199702051925.WAA11966@sinbin.demos.su> Subject: Re: bytebench not correct for SMP kernel ? To: smp@csn.net (Steve Passe) Date: Wed, 5 Feb 1997 22:25:20 +0300 (MSK) Cc: freebsd-smp@freebsd.org, mishania@demos.su In-Reply-To: <199702010316.UAA27071@clem.systemsix.com> from "Steve Passe" at Jan 31, 97 08:16:08 pm X-Mailer: ELM [version 2.4 PL24 ME7a] Content-Type: text Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > > Hi, > > [ nice answer by terry, thanx! ] > we test solaris x86 2.5 on the same hardware, it stable, but more slow that fbsd ... last bench in the table is on the sun Ultra1 167 Mhz, solaris 2.5, for comparising ... Alex. BYTE UNIX Benchmarks (Version 3.11) Asus P/I-P65UP5 with C-P6ND dual P6-200, adaptec 3940W, 256 Mb RAM ECC kernel/nosmp kernel/smp kernel/smp solaris 2.5 x86 sun Ultra1-167MHz smp_active=0 smp_active=2 (the same PC) solaris 2.5 Dhrystone 2 without register variables (10s, 6sampl) 445551.2 lps 448127.5 lps 448622.3 lps 293906.9 lps 260832.1 lps Dhrystone 2 using register variables (10s, 6sampl) 449567.0 lps 451046.9 lps 451203.7 lps 294847.9 lps 262159.9 lps Arithmetic Test (type = arithoh) (10s, 6sampl) 13243729.3 lps 13264989.4 lps 13268774.7 lps 910851.0 lps 1310960.5 lps Arithmetic Test (type = register) (10s, 6sampl) 54531.7 lps 54611.8 lps 54642.7 lps 50777.9 lps 17573.4 lps Arithmetic Test (type = short) (10s, 6sampl) 33325.9 lps 33383.2 lps 33395.7 lps 37920.8 lps 18206.3 lps Arithmetic Test (type = int) (10s, 6sampl) 54532.6 lps 54614.9 lps 54620.3 lps 50666.7 lps 17634.9 lps Arithmetic Test (type = long) (10s, 6sampl) 54533.5 lps 54583.9 lps 54660.0 lps 50543.0 lps 17624.9 lps Arithmetic Test (type = float) (10s, 6sampl) 60761.6 lps 60870.3 lps 60918.7 lps 48913.8 lps 29899.4 lps Arithmetic Test (type = double) (10s, 6sampl) 60779.7 lps 60865.2 lps 60931.6 lps 48745.1 lps 39952.6 lps System Call Overhead Test (10s, 6sampl) 68192.2 lps 51738.2 lps 38070.8 lps 42876.4 lps 49597.5 lps Pipe Throughput Test (10s, 6sampl) 92324.9 lps 68053.5 lps 57780.3 lps 60528.1 lps 45490.7 lps Pipe-based Context Switching Test (10s, 6sampl) 40542.8 lps 20177.0 lps 8785.4 lps 9760.9 lps 14541.3 lps Process Creation Test (10s, 6sampl) 3256.4 lps 2739.2 lps 1568.9 lps 486.1 lps 180.0 lps Execl Throughput Test ( 9s, 6sampl) 1437.4 lps 1206.6 lps 1032.5 lps 105.1 lps 68.9 lps File Read (10 seconds) (10s, 6sampl) 254626.0 KBps 190873.0 KBps 151645.0 KBps 45851.0 KBps 82114 KBps File Write (10 seconds) (10s, 6sampl) 3800.0 KBps 3800.0 KBps 3800.0 KBps 3691.0 KBps 1646 KBps File Copy (10 seconds) (10s, 6sampl) 3286.0 KBps 3286.0 KBps 3286.0 KBps 3810.0 KBps 2553 KBps File Read (30 seconds) (30s, 6sampl) 255236.0 KBps 191890.0 KBps 152978.0 KBps 43208.0 KBps 85191 KBps File Write (30 seconds) (30s, 6sampl) 3600.0 KBps 3600.0 KBps 3600.0 KBps 3866.0 KBps 1873 KBps File Copy (30 seconds) (30s, 6sampl) 1.0 KBps 3177.0 KBps 3236.0 KBps 3822.0 KBps 2724 KBps C Compiler Test (60s, 3sampl) 169.9 lpm 177.5 lpm 169.3 lpm 159.2 lpm 10042.0 lpm Shell scripts (1 concurrent) (60s, 3sampl) 270.3 lpm 293.9 lpm 264.0 lpm 137.3 lpm 90.7 lpm Shell scripts (2 concurrent) (60s, 3sampl) 149.0 lpm 170.6 lpm 150.3 lpm 93.3 lpm 52.7 lpm Shell scripts (4 concurrent) (60s, 3sampl) 80.3 lpm 94.3 lpm 83.0 lpm 54.3 lpm 30.0 lpm Shell scripts (8 concurrent) (60s, 3sampl) 40.3 lpm 40.0 lpm 42.0 lpm 30.0 lpm 15.0 lpm Dc: sqrt(2) to 99 decimal places (60s, 6sampl) 9533.5 lpm 8497.6 lpm 7406.6 lpm 2824.9 lpm 3283.1 lpm Recursion Test--Tower of Hanoi (10s, 6sampl) 5574.9 lps 5600.4 lps 5602.9 lps 5242.8 lps 5399.0 lps INDEX VALUES TEST INDEX INDEX INDEX INDEX INDEX Arithmetic Test (type = double) 23.9 23.9 24.0 19.2 15.7 Dhrystone 2 without register variables 19.9 20.0 20.1 13.1 11.7 Execl Throughput Test 87.1 73.1 62.6 6.4 4.2 File Copy (30 seconds) 0.0 17.7 18.1 21.4 15.2 Pipe-based Context Switching Test 30.7 15.3 6.7 7.4 11.0 Shell scripts (8 concurrent) 10.1 10.0 10.5 7.5 3.8 ========= ========= ========= ========= ========= SUM of 6 items 171.8 160.2 141.8 74.9 61.5 AVERAGE 28.6 26.7 23.6 12.5 10.3 From owner-freebsd-smp Wed Feb 5 11:57:11 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id LAA07315 for smp-outgoing; Wed, 5 Feb 1997 11:57:11 -0800 (PST) Received: from Sisyphos.MI.Uni-Koeln.DE (Sisyphos.MI.Uni-Koeln.DE [134.95.212.10]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id LAA07287; Wed, 5 Feb 1997 11:57:01 -0800 (PST) Received: from x14.mi.uni-koeln.de (annexr3-8.slip.Uni-Koeln.DE) by Sisyphos.MI.Uni-Koeln.DE with SMTP id AA11332 (5.67b/IDA-1.5); Wed, 5 Feb 1997 20:56:57 +0100 Received: (from se@localhost) by x14.mi.uni-koeln.de (8.8.5/8.6.9) id UAA05723; Wed, 5 Feb 1997 20:56:54 +0100 (CET) Message-Id: <19970205205654.SA58957@x14.mi.uni-koeln.de> Date: Wed, 5 Feb 1997 20:56:54 +0100 From: se@freebsd.org (Stefan Esser) To: regnauld@deepo.prosa.dk (Philippe Regnauld) Cc: freebsd-smp@freebsd.org, freebsd-hardware@freebsd.org Subject: Re: Finding Gigabyte 586-DX MB in Europe ? References: X-Mailer: Mutt 0.60-PL0 Mime-Version: 1.0 In-Reply-To: ; from Philippe Regnauld on Feb 4, 1997 17:14:32 +0100 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Feb 4, regnauld@deepo.prosa.dk (Philippe Regnauld) wrote: > .... Where might I find one of those critters, preferably in the Northern > half of Europe ? SMP is itching :-) I've just checked, and they are offered by the following German mail order companies on the web (all prices icnluding 15% VAT, multiply by 0.53 to convert into US$): Computer Profis Gigabyte 586 DX Dual Pentium, 512 KB Pipeline Burst, HX 485DM (US$257) Mint Data Gigabyte GA-586DX (512 Burst) 519DM (US$275) Sunshine 10519 Gigabyte 586 DX, 3.3V , Dual, Adaptec UW, 512 KB 509DM (US$270) They typcially charge some 30DM (US$20) for p&p ... Regards, STefan From owner-freebsd-smp Wed Feb 5 13:28:59 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id NAA13107 for smp-outgoing; Wed, 5 Feb 1997 13:28:59 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id NAA13010 for ; Wed, 5 Feb 1997 13:27:39 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id OAA00992 for ; Wed, 5 Feb 1997 14:25:59 -0700 Message-Id: <199702052125.OAA00992@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: smp@freebsd.org Subject: Re: paper I stumbled across In-reply-to: Your message of "Tue, 04 Feb 1997 23:05:08 EST." <199702050405.XAA03584@jenolan.caipgeneral> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 05 Feb 1997 14:25:59 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > > vger.rutgers.edu:/pub/linux/SMP/papers/netlocking.ps.gz > > It discusses the parallelization of the xkernel network protocol code > for shared memory multiprocessors. It includes analysis of various > simulations that the authors ran over their implementation using > various traces of tcp and udp connection sets. They also explore > strategies, payoff, and the overhead of the locking itself on the > operations that need to be performed. > I placed it on the FreeBSD SMP web site,down in the docs section of the top page. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Feb 5 14:10:35 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id OAA16005 for smp-outgoing; Wed, 5 Feb 1997 14:10:35 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id OAA15984 for ; Wed, 5 Feb 1997 14:10:29 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id PAA01184; Wed, 5 Feb 1997 15:06:35 -0700 Message-Id: <199702052206.PAA01184@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: bag@sinbin.demos.su (Alex G. Bulushev) cc: freebsd-smp@freebsd.org, mishania@demos.su Subject: Re: troubles with smp kernel In-reply-to: Your message of "Wed, 05 Feb 1997 22:10:44 +0300." <199702051910.WAA09819@sinbin.demos.su> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 05 Feb 1997 15:06:34 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >We try to run solaris x86 2.5 on this PC and it works without any problems, >after booting fbsd with kernel/smp it reboots. Average stable time about 1h, >maximum time before reboots about 6h. Now we use single 3940 and there is >no sharing irq's. We change our ram, and there is no parity error, we >change one CPU and now they have the same step = 6 ... The mptable looks fine. I wonder if you have corrupt sources? Another user reports grossly unstable system with current CTM sources. How/where/when did you get the SMP sources? Are you seeing any dying procs, sig10/11s, etc. while it is running? Are there any messages when it reboots? do you have DDB enabled in the config file, if not do so. --- We need to put a reference SMP binary onto the web page. I don't have a machine right now, could someone make an SMPGENERIC and place it on freefall? Please coordinate with me first so we don't have duplication of effort... -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Wed Feb 5 15:02:02 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id PAA20072 for smp-outgoing; Wed, 5 Feb 1997 15:02:02 -0800 (PST) Received: from kremvax.demos.su (kremvax.demos.su [194.87.0.20]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id OAA19912 for ; Wed, 5 Feb 1997 14:59:31 -0800 (PST) Received: by kremvax.demos.su (8.6.13/D) from 0@megillah.demos.su [194.87.0.21] with ESMTP id BAA26217; Thu, 6 Feb 1997 01:59:19 +0300 Received: by megillah.demos.su id BAA09073; (8.8.3/D) Thu, 6 Feb 1997 01:59:39 +0300 (MSK) Message-Id: <199702052259.BAA09073@megillah.demos.su> Subject: Re: troubles with smp kernel To: smp@csn.net (Steve Passe) Date: Thu, 6 Feb 1997 01:59:39 +0300 (MSK) Cc: bag@sinbin.demos.su, freebsd-smp@freebsd.org, mishania@demos.su In-Reply-To: <199702052206.PAA01184@clem.systemsix.com> from "Steve Passe" at Feb 5, 97 03:06:34 pm From: "Mikhail A. Sokolov" X-Class: Fast Organization: Demos Company, Ltd. Reply-To: mishania@demos.su X-Mailer: ELM [version 2.4 PL24 ME7a] Content-Type: text Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Hi, > > I wonder if you have corrupt sources? Another user reports grossly > unstable system with current CTM sources. How/where/when did you > get the SMP sources? I cvsuped them as of...~24/25/26 of January, with cvsup-file stolen from ~fsmp pages. After that, when all this mailing started, I rebuild everything once again on ~28.01 > Are you seeing any dying procs, sig10/11s, etc. while it is running? > Are there any messages when it reboots? do you have DDB enabled > in the config file, if not do so. Sometimes yes, there's some neat announce about catching floating exception on the console, sometimes it just reboots without declaring the war. Nothing in syslogd, and this exceptions belong to processor, - but well, we don't even touch the machine, just sitting couting when will it be down. Since it's already 6 processor we steal to check, it is not the faught of it, I guess. Not that the motherboard/brains are culprits: {skraldespand}/home/mishania/client-v5> uname -a SunOS skraldespand 5.5 Generic_103094-04 i86pc i386 i86pc {skraldespand}/home/mishania/client-v5> uptime 1:39am up 4:11, 1 user, load average: 1.97, 1.99, 1.91 Ah, skraldespand == fyllefrossa, it's the same PC. Not that I am going to bitch on instability of FreeBSD as opposed to Solaris, - never. What I can see it is probably already not hardware problem, since all this 4 hours the machine is up with solaris on board, it's very busy participating in DIC, RSA challange, in other words is having heavy computations. So, we have working processors, motherboard, brains, fatherboard (?;-) etc... Now I can only thank you for making us tune the beast (all these INT, etc), - now I think I am going to rebuild the SMP tree from scratch, getting fresh sources. I somehow guess they changed at least a bit in these past weeks? Ah, btw we now get this RAM PARITY CHECK ERROR back, so... Maybe I can already issue hints also: somehow there's an idea that 64mb SIMMs don't work with that ASUS we have (2xppro) ;-) Gotta continue playing construction worker ... > --- > We need to put a reference SMP binary onto the web page. I don't have > a machine right now, could someone make an SMPGENERIC and place it > on freefall? Please coordinate with me first so we don't have duplication > of effort... I can, but it will obviously never work correctly ;-) > > -- > Steve Passe | powered by > smp@csn.net | FreeBSD Thanks! -mishania From owner-freebsd-smp Wed Feb 5 16:21:05 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id QAA27472 for smp-outgoing; Wed, 5 Feb 1997 16:21:05 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id QAA27447 for ; Wed, 5 Feb 1997 16:20:52 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id RAA01926; Wed, 5 Feb 1997 17:16:02 -0700 Message-Id: <199702060016.RAA01926@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: mishania@demos.su cc: bag@sinbin.demos.su, freebsd-smp@freebsd.org Subject: Re: troubles with smp kernel In-reply-to: Your message of "Thu, 06 Feb 1997 01:59:39 +0300." <199702052259.BAA09073@megillah.demos.su> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 05 Feb 1997 17:16:01 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > > I wonder if you have corrupt sources? Another user reports grossly > > unstable system with current CTM sources. How/where/when did you > > get the SMP sources? > > I cvsuped them as of...~24/25/26 of January, with cvsup-file stolen from > ~fsmp pages. After that, when all this mailing started, I rebuild > everything once again on ~28.01 > > > Are you seeing any dying procs, sig10/11s, etc. while it is running? > > Are there any messages when it reboots? do you have DDB enabled > > in the config file, if not do so. > > Sometimes yes, there's some neat announce about catching floating exception > on the console, sometimes it just reboots without declaring the war. the root of your problem may be the floating point code. They is a major bug in the current SMP kernel where FP exception is almost guaranteed to crash the system! I've started up a page on the SMP site to collect info: http://www.freebsd.org/~fsmp/SMP/float.html on this page you will find a patxch that uses floating point emulation. I suggest adding this patch to see if it fixes the problem. --- > now I think I am going to rebuild the SMP tree from scratch, getting fresh > sources. I somehow guess they changed at least a bit in these past weeks? I guess it wouldn't hurt, but I suspect it won't solve the problem, I'm betting its the float issue. --- > Ah, btw we now get this RAM PARITY CHECK ERROR back, so... Maybe I can already > issue hints also: somehow there's an idea that 64mb SIMMs don't work with > that ASUS we have (2xppro) ;-) Gotta continue playing construction worker ... -- Steve Passe | powered by smp@csn.net | FreeBSD -----BEGIN PGP PUBLIC KEY BLOCK----- Version: 2.6.2 mQCNAzHe7tEAAAEEAM274wAEEdP+grIrV6UtBt54FB5ufifFRA5ujzflrvlF8aoE 04it5BsUPFi3jJLfvOQeydbegexspPXL6kUejYt2OeptHuroIVW5+y2M2naTwqtX WVGeBP6s2q/fPPAS+g+sNZCpVBTbuinKa/C4Q6HJ++M9AyzIq5EuvO0a8Rr9AAUR tBlTdGV2ZSBQYXNzZSA8c21wQGNzbi5uZXQ+ =ds99 -----END PGP PUBLIC KEY BLOCK----- From owner-freebsd-smp Wed Feb 5 17:07:47 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id RAA00631 for smp-outgoing; Wed, 5 Feb 1997 17:07:47 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.5/8.8.5) with SMTP id RAA00616 for ; Wed, 5 Feb 1997 17:07:41 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id SAA16154; Wed, 5 Feb 1997 18:04:31 -0700 From: Terry Lambert Message-Id: <199702060104.SAA16154@phaeton.artisoft.com> Subject: Re: screensavers broken? To: randyd@nconnect.net (Randy DuCharme) Date: Wed, 5 Feb 1997 18:04:31 -0700 (MST) Cc: terry@lambert.org, smp@freebsd.org In-Reply-To: <199702060035.SAA21746@atlantis.nconnect.net> from "Randy DuCharme" at Feb 5, 97 06:40:40 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > I didn't change the kernel tho', what I did was sup a fresh current source > tree and a > 'make world'. ( same difference tho ) I've kept the SMP and 'standard' > kernel > sources separate for fear of other problems, not to mention that it was > suggested > that I do so by Steve. This is what I do, too. > So, am I pretty much screwed until SMP kernel sources are updated, or do > you > think I can bring it back to life, and , if so, (it's not a big deal, I > can run a Uniprocessor kernel if I have to) how would I go about doing > that? Generally, I do the following: # cd /usr # mv src src-current # mkdir src # cd src # for i in /usr/src-current/* > do > ln -s $i . > done # rm sys # ln -s /b/src-smp/sys . Now /sys pounts to /usr/src/sys points to /b/src-smp/sys. Now I build everything, and everything gets their sys files from the SMP tree instead of the current tree. When I build the lkm's and 'ps' and 'w' and 'ifconfig', etc., etc., which all depend on kernel files, they build with the SMP headers instead of the -current headers. Everything then "just works". Regards, Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Thu Feb 6 14:56:59 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.5/8.8.5) id OAA12711 for smp-outgoing; Thu, 6 Feb 1997 14:56:59 -0800 (PST) Received: from bunyip.cc.uq.oz.au (daemon@bunyip.cc.uq.oz.au [130.102.2.1]) by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id OAA12705 for ; Thu, 6 Feb 1997 14:56:55 -0800 (PST) Received: (from daemon@localhost) by bunyip.cc.uq.oz.au (8.8.5/8.8.5) id IAA13386 for smp@freebsd.org; Fri, 7 Feb 1997 08:56:51 +1000 Received: from netfl15a.devetir.qld.gov.au by ogre.devetir.qld.gov.au (8.7.5/DEVETIR-E0.3a) with SMTP id IAA16058 for ; Fri, 7 Feb 1997 08:57:51 +1000 (EST) Received: from localhost by netfl15a.devetir.qld.gov.au (8.6.8.1/DEVETIR-0.1) id WAA00438 for ; Thu, 6 Feb 1997 22:58:29 GMT Message-Id: <199702062258.WAA00438@netfl15a.devetir.qld.gov.au> X-Mailer: exmh version 2.0beta 12/23/96 To: smp@freebsd.org Subject: What happens when FPU exceptions are masked? X-Face: 3}heU+2?b->-GSF-G4T4>jEB9~FR(V9lo&o>kAy=Pj&;oVOc<|pr%I/VSG"ZD32J>5gGC0N 7gj]^GI@M:LlqNd]|(2OxOxy@$6@/!,";-!OlucF^=jq8s57$%qXd/ieC8DhWmIy@J1AcnvSGV\|*! >Bvu7+0h4zCY^]{AxXKsDTlgA2m]fX$W@'8ev-Qi+-;%L'CcZ'NBL!@n?}q!M&Em3*eW7,093nOeV8 M)(u+6D;%B7j\XA/9j4!Gj~&jYzflG[#)E9sI&Xe9~y~Gn%fA7>F:YKr"Wx4cZU*6{^2ocZ!YyR Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 07 Feb 1997 08:58:28 +1000 From: Stephen Hocking Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Not having an SMP board to test this on, can anyone tell me what happens when one uses the setfpu functions to mask floating point errors (wand what happens when they do occur)? Stephen -- The views expressed above are not those of WorkCover Queensland, Australia. Unsolicited commercial email will be proofread at $900.00 US per message. Reading this constitutes acceptance of this condition.