From owner-freebsd-smp Sun Jan 5 05:48:58 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id FAA00173 for smp-outgoing; Sun, 5 Jan 1997 05:48:58 -0800 (PST) Received: from hda.hda.com (ip32-max1-fitch.ziplink.net [199.232.245.32]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id FAA00163; Sun, 5 Jan 1997 05:48:53 -0800 (PST) Received: (from dufault@localhost) by hda.hda.com (8.6.12/8.6.12) id IAA28862; Sun, 5 Jan 1997 08:44:55 -0500 From: Peter Dufault Message-Id: <199701051344.IAA28862@hda.hda.com> Subject: Re: atomic locking and In-Reply-To: <12453.852121399@critter.dk.tfs.com> from Poul-Henning Kamp at "Jan 1, 97 01:23:19 pm" To: phk@freebsd.org Date: Sun, 5 Jan 1997 08:44:54 -0500 (EST) Cc: smp@freebsd.org X-Mailer: ELM [version 2.4ME+ PL25 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Would it make sense to make a version of the various queues that had > built in locking ? Yes, but I think you should also package them up as functions with opaque structures. In addition to locking there is the issue of illegal access even with locking - consider naively traversing a circular queue and having the node you started with vanish off the queue even if each individual step is "properly" locked. Strict access to the queue structures will force a visit to each place they are used when moving over to the new interface. I also bet that these will grow more complicated anyway with items like object IDs for lock hierarchy and locker priority for priority boosting. -- Peter Dufault (dufault@hda.com) Realtime Machine Control and Simulation HD Associates, Inc. Voice: 508 433 6936 From owner-freebsd-smp Mon Jan 6 09:36:58 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA18059 for smp-outgoing; Mon, 6 Jan 1997 09:36:58 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id JAA18054 for ; Mon, 6 Jan 1997 09:36:52 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id KAA12039; Mon, 6 Jan 1997 10:27:22 -0700 From: Terry Lambert Message-Id: <199701061727.KAA12039@phaeton.artisoft.com> Subject: Re: Help To: smp@csn.net (Steve Passe) Date: Mon, 6 Jan 1997 10:27:22 -0700 (MST) Cc: Chad_Lafara@mmacmail.jccbi.gov, freebsd-smp@FreeBSD.ORG In-Reply-To: <199701021705.KAA05256@clem.systemsix.com> from "Steve Passe" at Jan 2, 97 10:05:52 am X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > > I wrote earlier about not having the opt_smp.h file. Well I also > > crash on: > > ioconf.h > > opt_sysvipc.h > > opt_param.h > > opt_cpu.h > > opt_smp_invltlb.h > > What have I done wrong? > > this looks like something more generic than just SMP configuration: > > opt_sysvipc.h > opt_param.h > opt_cpu.h > > are NOT SMP options and would break the non-SMP kernel if not found. Any more > clues? You are using an old "config" program. You need to rebuild your "config". Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Jan 6 09:40:00 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA18193 for smp-outgoing; Mon, 6 Jan 1997 09:40:00 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id JAA18163 for ; Mon, 6 Jan 1997 09:39:54 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id KAA12065; Mon, 6 Jan 1997 10:30:33 -0700 From: Terry Lambert Message-Id: <199701061730.KAA12065@phaeton.artisoft.com> Subject: Re: SMP patch for top To: randyd@nconnect.net (Randy DuCharme) Date: Mon, 6 Jan 1997 10:30:33 -0700 (MST) Cc: smp@FreeBSD.ORG In-Reply-To: <32CC8BD5.41C67EA6@nconnect.net> from "Randy DuCharme" at Jan 2, 97 10:32:21 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > Greetings, > > I've heard about it, but can't seem to locate it. Could someone point > me in the right direction ?? > > Are there any other SMP utility updates I should know about ?? :) pstat/mpstat. I don't know if anyone has written a formal list; I believe Steve Passe had a pstat that knew about multiple processors. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. From owner-freebsd-smp Mon Jan 6 10:27:13 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA20376 for smp-outgoing; Mon, 6 Jan 1997 10:27:13 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id KAA20371 for ; Mon, 6 Jan 1997 10:27:10 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id LAA03205; Mon, 6 Jan 1997 11:24:58 -0700 Message-Id: <199701061824.LAA03205@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Terry Lambert cc: randyd@nconnect.net (Randy DuCharme), smp@FreeBSD.ORG Subject: Re: SMP patch for top In-reply-to: Your message of "Mon, 06 Jan 1997 10:30:33 MST." <199701061730.KAA12065@phaeton.artisoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 06 Jan 1997 11:24:57 -0700 Sender: owner-smp@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk Hi, > > Are there any other SMP utility updates I should know about ?? :) > > pstat/mpstat. I don't know if anyone has written a formal list; I > believe Steve Passe had a pstat that knew about multiple processors. Not I, it was Peter who created patches for mpglue.c: >If anbody wants to try these out, they should add the counters for >the 4 ipi interrupts to 'vmstat -i' and 'systat -vmstat'. > >-Peter I am waiting for someone more CVS literate than I to create src/sys/tools and src/sys/docs directories in the SMP tree so that I can commit these and mptable.c. (hint, hint, Peter) -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Mon Jan 6 11:54:49 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA24045 for smp-outgoing; Mon, 6 Jan 1997 11:54:49 -0800 (PST) Received: from ping.at (pong.ping.at [193.81.13.2]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA24037 for ; Mon, 6 Jan 1997 11:54:40 -0800 (PST) Received: by ping.at id AA01919 (5.67b8/ping for ); Mon, 6 Jan 1997 20:54:22 +0100 Message-Id: <199701061954.AA01919@ping.at> Received: from ts1-11.oberwart.at.eu.net(193.83.148.11) by pong.ping.at via smap (V1.3) id smab01878; Mon Jan 6 20:54:13 1997 From: "Berhart Kristin" To: Subject: test - please ignore this email, okay? Thank you Date: Mon, 6 Jan 1997 20:52:02 +0100 X-Msmail-Priority: Normal X-Priority: 3 X-Mailer: Microsoft Internet Mail 4.70.1155 Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk Hello! I have some problems and so I have made this test. Please excuse, that IŽll have take your address. Buy, buy Kristin From owner-freebsd-smp Mon Jan 6 17:10:22 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id RAA17439 for smp-outgoing; Mon, 6 Jan 1997 17:10:22 -0800 (PST) Received: from po2.glue.umd.edu (root@po2.glue.umd.edu [129.2.128.45]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id RAA17434 for ; Mon, 6 Jan 1997 17:10:16 -0800 (PST) Received: from ginger.eng.umd.edu (ginger.eng.umd.edu [129.2.103.20]) by po2.glue.umd.edu (8.8.3/8.7.3) with ESMTP id UAA12071 for ; Mon, 6 Jan 1997 20:10:08 -0500 (EST) Received: from localhost (chuckr@localhost) by ginger.eng.umd.edu (8.8.3/8.7.3) with SMTP id UAA10516 for ; Mon, 6 Jan 1997 20:10:07 -0500 (EST) X-Authentication-Warning: ginger.eng.umd.edu: chuckr owned process doing -bs Date: Mon, 6 Jan 1997 20:10:07 -0500 (EST) From: Chuck Robey X-Sender: chuckr@ginger.eng.umd.edu To: FreeBSD-SMP@freebsd.org Subject: Starting off smp Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk I am trying to do the startup of smp on my machine. The directions on the FreeBSD smp web page aren't particularly helpful for two reasons: 1) I don't use sup or cvs-sup, I use ctm. 2) My main problem is interaction between my present cvs tree and the new, unrelated smp cvs tree. I tried going into my cvs-maintained /usr/src, and moving the sys directory there to sys-up (as suggested in the web page), then using the cvs smp tree I created via the ctm deltas to create a new sys. I've unpacked the smp ctm deltas into my home directory, /usr/chuckr/cvs1, so the commands I used to make the new smp sys were: cd /usr/src mv sys sys-UP cvs -d /usr/chuckr/cvs1 co sys I got this error message back: cvs: /home/ncvs value for CVS Root found in CVS/Root cvs: does not match command line -d /usr/chuckr/cvs1 setting cvs: you may wish to try the cvs command again without the -d option (note the CVSROOT for my main cvs tree is /home/ncvs). Anybody know what part of this I have wrong, so I can make my smp sys tree real? ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Wed Jan 8 19:15:24 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id TAA21861 for smp-outgoing; Wed, 8 Jan 1997 19:15:24 -0800 (PST) Received: from po2.glue.umd.edu (root@po2.glue.umd.edu [129.2.128.45]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id TAA21852 for ; Wed, 8 Jan 1997 19:15:19 -0800 (PST) Received: from modem.eng.umd.edu (modem.eng.umd.edu [129.2.98.187]) by po2.glue.umd.edu (8.8.3/8.7.3) with ESMTP id WAA01959 for ; Wed, 8 Jan 1997 22:15:16 -0500 (EST) Received: from localhost (chuckr@localhost) by modem.eng.umd.edu (8.8.3/8.7.3) with SMTP id WAA15099 for ; Wed, 8 Jan 1997 22:15:15 -0500 (EST) X-Authentication-Warning: modem.eng.umd.edu: chuckr owned process doing -bs Date: Wed, 8 Jan 1997 22:15:13 -0500 (EST) From: Chuck Robey X-Sender: chuckr@modem.eng.umd.edu To: FreeBSD-SMP@FreeBSD.org Subject: Testing new machine Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk I don't think I'm seeing any benefit from my second CPU. I've done some testing, compiling kernels, with different values given to make -j, to see the difference between response on my single processor machine versus my 2 CPU machine. Below are the times, as reported by /usr/bin/time, for a make on a rather complete kernel (no LKMs). The config files for the kernels on the two machines were identical (the generated kernel would not have run on both machines) and the sources were identical. I added a column "real/noralized" that is the value of the real time taken by the runs divided by the real time taken for the make -j 1 run, so as to give a comparative feeling. I am much more concerned with the relative values, and how they show what benefit I get from increasing the make -j value. I see little benefit from the extra CPU. Below is a mass of data, if you don't want to wade thru it, could you suggest some method I might use to actually verify my second CPU's utilization? The PPro machine is a Tyan Titan Pro. Both machines have identical disks, the PPro has 64 megs ram, the Pentium only 32 megs. Thanks for any hints. --(PPro/166, 2 CPUs, 512k internal cache/CPU)---------------------- real/normalized make -j 1, 420.00 real 200.32 user 27.67 sys 1.000 make -j 2, 311.71 real 200.86 user 28.95 sys 1.347 make -j 3, 282.59 real 202.50 user 28.66 sys 1.486 make -j 4, 275.37 real 203.65 user 28.64 sys 1.525 make -j 5, 271.67 real 204.25 user 28.83 sys 1.546 make -j 6, 263.22 real 203.66 user 29.88 sys 1.596 make -j 7, 260.99 real 204.40 user 29.53 sys 1.609 make -j 8, 261.07 real 204.93 user 29.36 sys 1.609 --(Pentium/166, 1 CPU, 512K external cache)------------------------ real/normalized make -j 1, 556.68 real 299.65 user 24.00 sys 1.000 make -j 2, 421.50 real 301.13 user 24.48 sys 1.321 make -j 3, 388.21 real 301.94 user 25.89 sys 1.434 make -j 4, 376.69 real 302.00 user 26.42 sys 1.478 make -j 5, 380.96 real 303.30 user 25.82 sys 1.461 make -j 6, 370.55 real 303.00 user 26.54 sys 1.502 make -j 7, 369.97 real 303.63 user 27.69 sys 1.505 make -j 8, 372.43 real 303.41 user 27.85 sys 1.495 I entered the values given from mptable in my 2 CPU machine, and that was the kernel that was being tested on that machine. Here is my dmesg. I don't know what to look for, for proof that my second CPU on my PPro system is being activated: California. All rights reserved. FreeBSD 3.0-SMP #0: Mon Jan 6 15:10:34 EST 1997 chuckr@picnic.cbr:/usr/src/sys/compile/CHUCKRSP FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 1, version: 0x00040011 cpu1 (AP): apic id: 0, version: 0x00040011 io0 (APIC): apic id: 2, version: 0x00170011 Calibrating clock(s) relative to mc146818A clock ... i8254 clock: 1193119 Hz CLK_USE_I8254_CALIBRATION not specified - using default frequency CPU: Pentium Pro (686-class CPU) Origin = "GenuineIntel" Id = 0x617 Stepping=7 Features=0xfbff,MTRR,PGE,MCA,CMOV> real memory = 67108864 (65536K bytes) avail memory = 62586880 (61120K bytes) pcibus_setup(1): mode 1 addr port (0x0cf8) is 0x8000005c pcibus_setup(1a): mode1res=0x80000000 (0x80000000) pcibus_check: device 0 is there (id=12378086) Probing for devices on PCI bus 0: configuration mode 1 allows 32 devices. chip0 rev 2 on pci0:0 chip1 rev 1 on pci0:7:0 chip2 rev 0 on pci0:7:1 mapreg[20] type=1 addr=00003000 size=0010. ncr0 rev 19 int a irq 18 on pci0:12 Freeing (NOT implimented) irq 11 for ISA cards. mapreg[10] type=1 addr=00006000 size=0100. mapreg[14] type=0 addr=e0000000 size=0100. mapreg[18] type=0 addr=e0001000 size=1000. reg20: virtual=0xf6fed000 physical=0xe0000000 size=0x100 ncr0: restart (scsi reset). BIOS values: dmode: 8e, dcntl: a1, ctest3: 31 dmode: ce/8e, dcntl: a1/a1, ctest3: 01/31 ncr0 scanning for targets 0..6 and 8..15 (V2 pl24 96/12/14) ncr0 waiting for scsi devices to settle (ncr0:0:0): "CONNER CP30200 SUN0207 4544" type 0 fixed SCSI 2 sd0(ncr0:0:0): Direct-Access sd0(ncr0:0:0): 5.0 MB/s (200 ns, offset 16) 203MB (416108 512 byte sectors) sd0(ncr0:0:0): with 2123 cyls, 4 heads, and an average 49 sectors/track (ncr0:1:0): "DEC DSP3210S 442A" type 0 fixed SCSI 2 sd1(ncr0:1:0): Direct-Access sd1(ncr0:1:0): 10.0 MB/s (100 ns, offset 15) 2047MB (4194303 512 byte sectors) sd1(ncr0:1:0): with 3045 cyls, 16 heads, and an average 86 sectors/track (ncr0:2:0): "DEC DSP3210D 442Y" type 0 fixed SCSI 2 sd2(ncr0:2:0): Direct-Access sd2(ncr0:2:0): 10.0 MB/s (100 ns, offset 15) 2047MB (4194303 512 byte sectors) sd2(ncr0:2:0): with 3045 cyls, 16 heads, and an average 86 sectors/track (ncr0:4:0): "TEXEL CD-ROM DM-XX28 3.05" type 5 removable SCSI 2 cd0(ncr0:4:0): CD-ROM cd0(ncr0:4:0): asynchronous. cd present [325252 x 2048 byte records] pci0: uses 4352 bytes of memory from e0000000 upto e0001fff. pci0: uses 272 bytes of I/O space from 3000 upto 60ff. Probing for devices on the ISA bus: sc0 at 0x60-0x6f irq 1 on motherboard sc0: VGA color <16 virtual consoles, flags=0x0> ed0 at 0x280-0x29f irq 5 maddr 0xd8000 msize 16384 on isa ed0: address 00:00:c0:e8:15:52, type WD8013WC (16 bit) bpf: ed0 attached sio0 at 0x3f8-0x3ff irq 4 on isa sio0: type 16550A sio1 at 0x2f8-0x2ff irq 3 on isa sio1: type 16550A lpt0 at 0x378-0x37f irq 7 on isa lpt0: Interrupt-driven port lp0: TCP/IP capable interface bpf: lp0 attached fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa fdc0: NEC 72065B fd0: 1.44MB 3.5in fd1: 1.2MB 5.25in npx0 on motherboard npx0: INT 16 interface imasks: bio c0040040, tty f00000ba, net f00000ba BIOS Geometries: 0:00ca3f20 0..202=203 cylinders, 0..63=64 heads, 1..32=32 sectors 1:02077f3f 0..519=520 cylinders, 0..127=128 heads, 1..63=63 sectors 2:02077f3f 0..519=520 cylinders, 0..127=128 heads, 1..63=63 sectors 0 accounted for Device configuration finished. Considering FFS root f/s. configure() finished. Enabled INTs: 1, 2, 3, 4, 5, 6, 7, 8, 18, imen: 0x00fbfe01 new masks: bio c0040040, tty f00000ba, net f00000ba bpf: tun0 attached bpf: sl0 attached bpf: ppp0 attached bpf: lo0 attached sd1s1: type 0xa5, start 63, end = 4193279, size 4193217 : OK SMP: All idle procs online. sd2s1: type 0xa5, start 63, end = 4193279, size 4193217 : OK ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Thu Jan 9 06:55:02 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id GAA02825 for smp-outgoing; Thu, 9 Jan 1997 06:55:02 -0800 (PST) Received: from june.cs.washington.edu (june.cs.washington.edu [128.95.1.4]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id GAA02820 for ; Thu, 9 Jan 1997 06:55:00 -0800 (PST) Received: from slownet (h232.dyn.cs.washington.edu [128.95.8.232]) by june.cs.washington.edu (8.8.3+CSE/7.2ju) with ESMTP id GAA27078 for ; Thu, 9 Jan 1997 06:54:51 -0800 Message-ID: <32D41F66.3941@cs.washington.edu> Date: Wed, 08 Jan 1997 18:27:50 -0400 From: "Marc E. Fiuczynski" Organization: UW Dept. of Computer Science and Engr. X-Sender: "Marc E. Fiuczynski" X-Mailer: Mozilla 4.0b1 (Win95; I) MIME-Version: 1.0 To: smp@freebsd.org Subject: unusual question X-Priority: Normal Content-Type: multipart/alternative; boundary="----------695161C641C12" Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk ------------695161C641C12 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Hi, We've developed our own operating system called SPIN (which was designed to be dynamically extensible by arbitrary users). All of the poks (plain old kernel services) are implemented by our OS (threads, all but the lowest level of VM, dynamic linking and extensibility support) and other services (such as filesystems, networking, a unix emulation library) are implemented by extensions that are dynamically linked into the kernel's virtual address space. Virtual all of our system is implemented in Modula-3. The one thing that we didn't implement ourselves are device drivers for the various i/o cards. For the alpha workstation we borrow device drivers from DEC OSF/1. For the x86, well you guessed it, we borrowed device drivers from FreeBSD 2.1.6. In terms of device driver support we use some of the files in i386/* and pci/* and of course various header files. We are now interested to run SPIN on multiprocs using our FreeBSD 2.1.6 driver base. I'm curious which device drivers are affected by the SMP code. Could someone tell me which doc I should read to find out which components of FreeBSD are in particular heavily modified for SMP support. My guess is some of the VM and locore stuff. I heard mentions of "APIC" and would appreciate if someone pointed me at the proper documentation for that as well, as I have no clue what that is (though I guess it is some board support for multiprocs). For those of you curious about SPIN please take a look at http://www.cs.washington.edu/research/projects/spin/www you can also get there from my home page by following the SPIN link http://www.cs.washington.edu/homes/mef Finally, if you have any questions about SPIN please fell free to contact me. Marc ------------695161C641C12 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii
Hi, 
 
We've developed our own operating system called SPIN (which was designed to be dynamically extensible by arbitrary users).  All of the poks (plain old kernel services) are implemented by our OS (threads, all but the lowest level of VM, dynamic linking and extensibility support) and other services (such as filesystems, networking, a unix emulation library) are implemented by extensions that are dynamically linked into the kernel's virtual address space.  Virtual all of our system is implemented in Modula-3.
 
The one thing that we didn't implement ourselves are device drivers for the various i/o cards.  For the alpha workstation we borrow device drivers from DEC OSF/1.  For the x86, well you guessed it, we borrowed device drivers from FreeBSD 2.1.6.  In terms of device driver support we use some of the files in i386/* and pci/* and of course various header files.
 
We are now interested to run SPIN on multiprocs using our FreeBSD 2.1.6 driver base.  I'm curious which device drivers are affected by the SMP code.  Could someone tell me which doc I should read to find out which components of FreeBSD are in particular heavily modified for SMP support.  My guess is some of the VM and locore stuff.  I heard mentions of "APIC" and would appreciate if someone pointed me at the proper documentation for that as well, as I have no clue what that is (though I guess it is some board support for multiprocs).
 
For those of you curious about SPIN please take a look at 
http://www.cs.washington.edu/research/projects/spin/www
you can also get there from my home page by following the SPIN link
http://www.cs.washington.edu/homes/mef
 
Finally, if you have any questions about SPIN please fell free to contact me.
 
Marc
 
------------695161C641C12-- From owner-freebsd-smp Thu Jan 9 07:05:01 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id HAA03323 for smp-outgoing; Thu, 9 Jan 1997 07:05:01 -0800 (PST) Received: from ihgw2.lucent.com (ihgw2.lucent.com [207.19.48.2]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id HAA03305 for ; Thu, 9 Jan 1997 07:04:45 -0800 (PST) From: ejc@naserver1.cb.lucent.com Received: from nasvr1.cb.lucent.com by ihig2.firewall.lucent.com (SMI-8.6/EMS-L sol2) id IAA07411; Thu, 9 Jan 1997 08:14:16 -0600 Received: from barnacle.cb.lucent.com by nasvr1.cb.lucent.com (5.x/EMS-L sol2) id AA01711; Thu, 9 Jan 1997 09:20:57 -0500 Received: by barnacle.cb.lucent.com (SMI-8.6/EMS-1.1 Sol2) id JAA26795; Thu, 9 Jan 1997 09:22:26 -0500 Date: Thu, 9 Jan 1997 09:22:26 -0500 Message-Id: <199701091422.JAA26795@barnacle.cb.lucent.com> To: FreeBSD-SMP@freebsd.org, chuckr@Glue.umd.edu Subject: Re: Testing new machine X-Sun-Charset: US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hello I didn't see in your dmesg that the second cpu was turned on. By chance did you "sysctl -w kern.smp_active=2" Peace, Eric J. Chet - ejc@nasvr1.cb.lucent.com - ejc@bazzle.com > From nauucp Wed Jan 8 22:27 EST 1997 > X-Authentication-Warning: modem.eng.umd.edu: chuckr owned process doing -bs > From: Chuck Robey > X-Sender: chuckr@modem.eng.umd.edu > To: FreeBSD-SMP@freebsd.org > Subject: Testing new machine > Mime-Version: 1.0 > X-Loop: FreeBSD.org > > I don't think I'm seeing any benefit from my second CPU. > I've done some testing, compiling kernels, with different > values given to make -j, to see the difference between > response on my single processor machine versus my 2 CPU > machine. > > Below are the times, as reported by /usr/bin/time, for > a make on a rather complete kernel (no LKMs). The config > files for the kernels on the two machines were identical > (the generated kernel would not have run on both machines) > and the sources were identical. I added a column > "real/noralized" that is the value of the real time taken > by the runs divided by the real time taken for the make -j 1 > run, so as to give a comparative feeling. > > I am much more concerned with the relative values, and how they show what > benefit I get from increasing the make -j value. I see little benefit > from the extra CPU. > > Below is a mass of data, if you don't want to wade thru it, could you > suggest some method I might use to actually verify my second CPU's > utilization? The PPro machine is a Tyan Titan Pro. Both machines have > identical disks, the PPro has 64 megs ram, the Pentium only 32 megs. > > Thanks for any hints. > > --(PPro/166, 2 CPUs, 512k internal cache/CPU)---------------------- > real/normalized > make -j 1, 420.00 real 200.32 user 27.67 sys 1.000 > make -j 2, 311.71 real 200.86 user 28.95 sys 1.347 > make -j 3, 282.59 real 202.50 user 28.66 sys 1.486 > make -j 4, 275.37 real 203.65 user 28.64 sys 1.525 > make -j 5, 271.67 real 204.25 user 28.83 sys 1.546 > make -j 6, 263.22 real 203.66 user 29.88 sys 1.596 > make -j 7, 260.99 real 204.40 user 29.53 sys 1.609 > make -j 8, 261.07 real 204.93 user 29.36 sys 1.609 > > --(Pentium/166, 1 CPU, 512K external cache)------------------------ > real/normalized > make -j 1, 556.68 real 299.65 user 24.00 sys 1.000 > make -j 2, 421.50 real 301.13 user 24.48 sys 1.321 > make -j 3, 388.21 real 301.94 user 25.89 sys 1.434 > make -j 4, 376.69 real 302.00 user 26.42 sys 1.478 > make -j 5, 380.96 real 303.30 user 25.82 sys 1.461 > make -j 6, 370.55 real 303.00 user 26.54 sys 1.502 > make -j 7, 369.97 real 303.63 user 27.69 sys 1.505 > make -j 8, 372.43 real 303.41 user 27.85 sys 1.495 > > I entered the values given from mptable in my 2 CPU machine, > and that was the kernel that was being tested on that machine. > Here is my dmesg. I don't know what to look for, for proof > that my second CPU on my PPro system is being activated: > > > California. All rights reserved. > > FreeBSD 3.0-SMP #0: Mon Jan 6 15:10:34 EST 1997 > chuckr@picnic.cbr:/usr/src/sys/compile/CHUCKRSP > FreeBSD/SMP: Multiprocessor motherboard > cpu0 (BSP): apic id: 1, version: 0x00040011 > cpu1 (AP): apic id: 0, version: 0x00040011 > io0 (APIC): apic id: 2, version: 0x00170011 > Calibrating clock(s) relative to mc146818A clock ... i8254 clock: 1193119 Hz > CLK_USE_I8254_CALIBRATION not specified - using default frequency > CPU: Pentium Pro (686-class CPU) > Origin = "GenuineIntel" Id = 0x617 Stepping=7 > Features=0xfbff,MTRR,PGE,MCA,CMOV> > real memory = 67108864 (65536K bytes) > avail memory = 62586880 (61120K bytes) > pcibus_setup(1): mode 1 addr port (0x0cf8) is 0x8000005c > pcibus_setup(1a): mode1res=0x80000000 (0x80000000) > pcibus_check: device 0 is there (id=12378086) > Probing for devices on PCI bus 0: > configuration mode 1 allows 32 devices. > chip0 rev 2 on pci0:0 > chip1 rev 1 on pci0:7:0 > chip2 rev 0 on pci0:7:1 > mapreg[20] type=1 addr=00003000 size=0010. > ncr0 rev 19 int a irq 18 on pci0:12 > Freeing (NOT implimented) irq 11 for ISA cards. > mapreg[10] type=1 addr=00006000 size=0100. > mapreg[14] type=0 addr=e0000000 size=0100. > mapreg[18] type=0 addr=e0001000 size=1000. > reg20: virtual=0xf6fed000 physical=0xe0000000 size=0x100 > ncr0: restart (scsi reset). > BIOS values: dmode: 8e, dcntl: a1, ctest3: 31 > dmode: ce/8e, dcntl: a1/a1, ctest3: 01/31 > ncr0 scanning for targets 0..6 and 8..15 (V2 pl24 96/12/14) > ncr0 waiting for scsi devices to settle > (ncr0:0:0): "CONNER CP30200 SUN0207 4544" type 0 fixed SCSI 2 > sd0(ncr0:0:0): Direct-Access > sd0(ncr0:0:0): 5.0 MB/s (200 ns, offset 16) > 203MB (416108 512 byte sectors) > sd0(ncr0:0:0): with 2123 cyls, 4 heads, and an average 49 sectors/track > (ncr0:1:0): "DEC DSP3210S 442A" type 0 fixed SCSI 2 > sd1(ncr0:1:0): Direct-Access > sd1(ncr0:1:0): 10.0 MB/s (100 ns, offset 15) > 2047MB (4194303 512 byte sectors) > sd1(ncr0:1:0): with 3045 cyls, 16 heads, and an average 86 sectors/track > (ncr0:2:0): "DEC DSP3210D 442Y" type 0 fixed SCSI 2 > sd2(ncr0:2:0): Direct-Access > sd2(ncr0:2:0): 10.0 MB/s (100 ns, offset 15) > 2047MB (4194303 512 byte sectors) > sd2(ncr0:2:0): with 3045 cyls, 16 heads, and an average 86 sectors/track > (ncr0:4:0): "TEXEL CD-ROM DM-XX28 3.05" type 5 removable SCSI 2 > cd0(ncr0:4:0): CD-ROM > cd0(ncr0:4:0): asynchronous. > cd present [325252 x 2048 byte records] > pci0: uses 4352 bytes of memory from e0000000 upto e0001fff. > pci0: uses 272 bytes of I/O space from 3000 upto 60ff. > Probing for devices on the ISA bus: > sc0 at 0x60-0x6f irq 1 on motherboard > sc0: VGA color <16 virtual consoles, flags=0x0> > ed0 at 0x280-0x29f irq 5 maddr 0xd8000 msize 16384 on isa > ed0: address 00:00:c0:e8:15:52, type WD8013WC (16 bit) > bpf: ed0 attached > sio0 at 0x3f8-0x3ff irq 4 on isa > sio0: type 16550A > sio1 at 0x2f8-0x2ff irq 3 on isa > sio1: type 16550A > lpt0 at 0x378-0x37f irq 7 on isa > lpt0: Interrupt-driven port > lp0: TCP/IP capable interface > bpf: lp0 attached > fdc0 at 0x3f0-0x3f7 irq 6 drq 2 on isa > fdc0: NEC 72065B > fd0: 1.44MB 3.5in > fd1: 1.2MB 5.25in > npx0 on motherboard > npx0: INT 16 interface > imasks: bio c0040040, tty f00000ba, net f00000ba > BIOS Geometries: > 0:00ca3f20 0..202=203 cylinders, 0..63=64 heads, 1..32=32 sectors > 1:02077f3f 0..519=520 cylinders, 0..127=128 heads, 1..63=63 sectors > 2:02077f3f 0..519=520 cylinders, 0..127=128 heads, 1..63=63 sectors > 0 accounted for > Device configuration finished. > Considering FFS root f/s. > configure() finished. > Enabled INTs: 1, 2, 3, 4, 5, 6, 7, 8, 18, imen: 0x00fbfe01 > new masks: bio c0040040, tty f00000ba, net f00000ba > bpf: tun0 attached > bpf: sl0 attached > bpf: ppp0 attached > bpf: lo0 attached > sd1s1: type 0xa5, start 63, end = 4193279, size 4193217 : OK > SMP: All idle procs online. > sd2s1: type 0xa5, start 63, end = 4193279, size 4193217 : OK > > > ----------------------------+----------------------------------------------- > Chuck Robey | Interests include any kind of voice or data > chuckr@eng.umd.edu | communications topic, C programming, and Unix. > 9120 Edmonston Ct #302 | > Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD > (301) 220-2114 | version 3.0 current -- and great FUN! > ----------------------------+----------------------------------------------- > > > > From owner-freebsd-smp Thu Jan 9 07:11:36 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id HAA03514 for smp-outgoing; Thu, 9 Jan 1997 07:11:36 -0800 (PST) Received: from tfs.com (tfs.com [140.145.250.1]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id HAA03507 for ; Thu, 9 Jan 1997 07:11:33 -0800 (PST) Received: from schizo.dk.tfs.com by tfs.com (smail3.1.28.1) with SMTP id m0viM88-0003wdC; Thu, 9 Jan 97 07:10 PST Received: from critter.dk.tfs.com (critter.dk.tfs.com [140.145.230.252]) by schizo.dk.tfs.com (8.8.2/8.7.3) with ESMTP id QAA09828; Thu, 9 Jan 1997 16:10:54 +0100 (MET) Received: from critter.dk.tfs.com (localhost [127.0.0.1]) by critter.dk.tfs.com (8.8.2/8.8.2) with ESMTP id QAA07516; Thu, 9 Jan 1997 16:11:04 +0100 (MET) To: "Marc E. Fiuczynski" cc: smp@freebsd.org Subject: Re: unusual question In-reply-to: Your message of "Wed, 08 Jan 1997 18:27:50 -0400." <32D41F66.3941@cs.washington.edu> Date: Thu, 09 Jan 1997 16:11:03 +0100 Message-ID: <7514.852822663@critter.dk.tfs.com> From: Poul-Henning Kamp Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk >We are now interested to run SPIN on multiprocs using our FreeBSD 2.1.6 >driver base. I'm curious which device drivers are affected by the SMP >code. So far we have not modified any device-drivers for SMP operation. This is only possible because we are still running in the OBULATEK mode ("One Big Ugly Lock Around The Entire Kernel"). >From prior experience, I would think that device drivers will be fitted with a LOCK each, so that they stay unchanged, until such time as somebody fits them with fine-grain locking. Generally, you can allow device-drivers to be multi-threaded only as far as you have separate pieces of hardware. Ie, in you average FreeBSD system you could have one thread/cpu per sio port, but only one for wdc0+wd0+wd1 because they share registers. I would expect the "one-lock per device-driver" to be implemented in the code that calls into the device drivers, simply to avoid changing all of them, so most device-drivers will probably not change much if at all. -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. http://www.freebsd.org/~phk | phk@login.dknet.dk Private mailbox. whois: [PHK] | phk@tfs.com TRW Financial Systems, Inc. Power and ignorance is a disgusting cocktail. From owner-freebsd-smp Thu Jan 9 08:44:31 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id IAA08983 for smp-outgoing; Thu, 9 Jan 1997 08:44:31 -0800 (PST) Received: from po2.glue.umd.edu (root@po2.glue.umd.edu [129.2.128.45]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id IAA08978 for ; Thu, 9 Jan 1997 08:44:27 -0800 (PST) Received: from packet.eng.umd.edu (packet.eng.umd.edu [129.2.98.184]) by po2.glue.umd.edu (8.8.3/8.7.3) with ESMTP id LAA12247; Thu, 9 Jan 1997 11:44:24 -0500 (EST) Received: from localhost (chuckr@localhost) by packet.eng.umd.edu (8.8.3/8.7.3) with SMTP id LAA07780; Thu, 9 Jan 1997 11:44:23 -0500 (EST) X-Authentication-Warning: packet.eng.umd.edu: chuckr owned process doing -bs Date: Thu, 9 Jan 1997 11:44:23 -0500 (EST) From: Chuck Robey X-Sender: chuckr@packet.eng.umd.edu To: ejc@naserver1.cb.lucent.com cc: FreeBSD-SMP@freebsd.org Subject: Re: Testing new machine In-Reply-To: <199701091422.JAA26795@barnacle.cb.lucent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Thu, 9 Jan 1997 ejc@naserver1.cb.lucent.com wrote: > Hello > > I didn't see in your dmesg that the second cpu was turned on. > By chance did you "sysctl -w kern.smp_active=2" > You and Steve both caught that, and it sure was the problem. Right now I'm making a set of 2 CPU benchmarks, and they show definite improvements. I suppose I should put the sysctl somewhere in my rc file ... is there any particular place in there (has anything GOT to happen first) ? > > Peace, > Eric J. Chet > - ejc@nasvr1.cb.lucent.com > - ejc@bazzle.com > > ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Thu Jan 9 09:24:50 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA11144 for smp-outgoing; Thu, 9 Jan 1997 09:24:50 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id JAA11126 for ; Thu, 9 Jan 1997 09:24:46 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id KAA22487; Thu, 9 Jan 1997 10:24:02 -0700 Message-Id: <199701091724.KAA22487@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Chuck Robey cc: FreeBSD-SMP@freebsd.org Subject: Re: Testing new machine In-reply-to: Your message of "Thu, 09 Jan 1997 11:44:23 EST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 09 Jan 1997 10:24:01 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, >I suppose I should put the sysctl somewhere in my rc file ... is there any >particular place in there (has anything GOT to happen first) ? I'd suggest /usr/local/etc/rc.d/smp.sh -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Jan 9 09:32:02 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA11609 for smp-outgoing; Thu, 9 Jan 1997 09:32:02 -0800 (PST) Received: from po1.glue.umd.edu (root@po1.glue.umd.edu [129.2.128.44]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id JAA11582 for ; Thu, 9 Jan 1997 09:31:58 -0800 (PST) Received: from packet.eng.umd.edu (packet.eng.umd.edu [129.2.98.184]) by po1.glue.umd.edu (8.8.3/8.7.3) with ESMTP id MAA02158; Thu, 9 Jan 1997 12:31:50 -0500 (EST) Received: from localhost (chuckr@localhost) by packet.eng.umd.edu (8.8.3/8.7.3) with SMTP id MAA07852; Thu, 9 Jan 1997 12:31:49 -0500 (EST) X-Authentication-Warning: packet.eng.umd.edu: chuckr owned process doing -bs Date: Thu, 9 Jan 1997 12:31:49 -0500 (EST) From: Chuck Robey X-Sender: chuckr@packet.eng.umd.edu To: Steve Passe cc: FreeBSD-SMP@freebsd.org Subject: Re: Testing new machine In-Reply-To: <199701091724.KAA22487@clem.systemsix.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Thu, 9 Jan 1997, Steve Passe wrote: > Hi, > > >I suppose I should put the sysctl somewhere in my rc file ... is there any > >particular place in there (has anything GOT to happen first) ? > > I'd suggest /usr/local/etc/rc.d/smp.sh Thanks, Steve, but I wasn't looking for a place to declare it as much as an indication of anything that happens in /etc/rc that HAS to take place first. I figure I have to hook it in, in either /etc/rc, /etc/rc.i386 or /etc/rc.local (which could very well call /usr/local/etc/rc.d/smp.sh) but I don't know if anything in the present startup sequence is required before turning on multiprocessing. > -- > Steve Passe | powered by > smp@csn.net | FreeBSD > > ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Thu Jan 9 09:36:17 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA11836 for smp-outgoing; Thu, 9 Jan 1997 09:36:17 -0800 (PST) Received: from cbgw1.lucent.com (cbgw1.lucent.com [192.20.239.133]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id JAA11831 for ; Thu, 9 Jan 1997 09:36:13 -0800 (PST) From: ejc@naserver1.cb.lucent.com Received: from nasvr1.cb.lucent.com by cbig1.firewall.lucent.com (SMI-8.6/EMS-L sol2) id MAA11822; Thu, 9 Jan 1997 12:29:06 -0500 Received: from barnacle.cb.lucent.com by nasvr1.cb.lucent.com (5.x/EMS-L sol2) id AA22693; Thu, 9 Jan 1997 12:37:22 -0500 Received: by barnacle.cb.lucent.com (SMI-8.6/EMS-1.1 Sol2) id MAA27883; Thu, 9 Jan 1997 12:38:49 -0500 Date: Thu, 9 Jan 1997 12:38:49 -0500 Message-Id: <199701091738.MAA27883@barnacle.cb.lucent.com> To: ejc@naserver1.cb.lucent.com, chuckr@Glue.umd.edu Subject: Re: Testing new machine Cc: FreeBSD-SMP@freebsd.org X-Sun-Charset: US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > From nauucp Thu Jan 9 12:06 EST 1997 > Cc: FreeBSD-SMP@freebsd.org > X-Authentication-Warning: packet.eng.umd.edu: chuckr owned process doing -bs > From: Chuck Robey > X-Sender: chuckr@packet.eng.umd.edu > To: ejc@naserver1.cb.lucent.com > Original-Cc: FreeBSD-SMP@freebsd.org > Subject: Re: Testing new machine > Mime-Version: 1.0 > X-Loop: FreeBSD.org > > On Thu, 9 Jan 1997 ejc@naserver1.cb.lucent.com wrote: > > > Hello > > > > I didn't see in your dmesg that the second cpu was turned on. > > By chance did you "sysctl -w kern.smp_active=2" > > > > You and Steve both caught that, and it sure was the problem. Right now > I'm making a set of 2 CPU benchmarks, and they show definite improvements. > > I suppose I should put the sysctl somewhere in my rc file ... is there any > particular place in there (has anything GOT to happen first) ? Chuck I think your safe to put it in /etc/rc.local. AFAIK everything happens before we get to /etc/rc. Peace, Eric J. Chet - ejc@nasvr1.cb.lucent.com - ejc@bazzle.com From owner-freebsd-smp Thu Jan 9 09:38:06 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA11964 for smp-outgoing; Thu, 9 Jan 1997 09:38:06 -0800 (PST) Received: from po1.glue.umd.edu (root@po1.glue.umd.edu [129.2.128.44]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id JAA11959 for ; Thu, 9 Jan 1997 09:38:04 -0800 (PST) Received: from packet.eng.umd.edu (packet.eng.umd.edu [129.2.98.184]) by po1.glue.umd.edu (8.8.3/8.7.3) with ESMTP id MAA02370; Thu, 9 Jan 1997 12:37:56 -0500 (EST) Received: from localhost (chuckr@localhost) by packet.eng.umd.edu (8.8.3/8.7.3) with SMTP id MAA07860; Thu, 9 Jan 1997 12:37:55 -0500 (EST) X-Authentication-Warning: packet.eng.umd.edu: chuckr owned process doing -bs Date: Thu, 9 Jan 1997 12:37:55 -0500 (EST) From: Chuck Robey X-Sender: chuckr@packet.eng.umd.edu To: ejc@naserver1.cb.lucent.com cc: FreeBSD-SMP@freebsd.org Subject: Re: Testing new machine In-Reply-To: <199701091738.MAA27883@barnacle.cb.lucent.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk On Thu, 9 Jan 1997 ejc@naserver1.cb.lucent.com wrote: > > > > > > I didn't see in your dmesg that the second cpu was turned on. > > > By chance did you "sysctl -w kern.smp_active=2" > > > > > > > You and Steve both caught that, and it sure was the problem. Right now > > I'm making a set of 2 CPU benchmarks, and they show definite improvements. > > > > I suppose I should put the sysctl somewhere in my rc file ... is there any > > particular place in there (has anything GOT to happen first) ? > > Chuck > I think your safe to put it in /etc/rc.local. AFAIK everything happens > before we get to /etc/rc. Thanks, that's what I was hunting. SMP is fun! ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Thu Jan 9 09:53:04 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id JAA12382 for smp-outgoing; Thu, 9 Jan 1997 09:53:04 -0800 (PST) Received: from po2.glue.umd.edu (root@po2.glue.umd.edu [129.2.128.45]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id JAA12376 for ; Thu, 9 Jan 1997 09:53:01 -0800 (PST) Received: from packet.eng.umd.edu (packet.eng.umd.edu [129.2.98.184]) by po2.glue.umd.edu (8.8.3/8.7.3) with ESMTP id MAA14175 for ; Thu, 9 Jan 1997 12:52:59 -0500 (EST) Received: from localhost (chuckr@localhost) by packet.eng.umd.edu (8.8.3/8.7.3) with SMTP id MAA07897 for ; Thu, 9 Jan 1997 12:52:57 -0500 (EST) X-Authentication-Warning: packet.eng.umd.edu: chuckr owned process doing -bs Date: Thu, 9 Jan 1997 12:52:57 -0500 (EST) From: Chuck Robey X-Sender: chuckr@packet.eng.umd.edu To: FreeBSD-SMP@FreeBSD.org Subject: New stats Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk Got my new statistics now, from the Titan Pro 2 CPU system, 64M ram, PPro 166 with 512K cache. First set of specs is without the second CPU turned on, second is with both CPUs active. This is cooking! Last set is from my Pentium system, just for comparison, they were all done compiling the exact same kernel sources, same config file. --(PPro/166, 2 CPUs, 512k internal cache/CPU)---------------------- 1 CPU Active real/normalized make -j 1, 420.00 real 200.32 user 27.67 sys 1.000 make -j 2, 311.71 real 200.86 user 28.95 sys 1.347 make -j 3, 282.59 real 202.50 user 28.66 sys 1.486 make -j 4, 275.37 real 203.65 user 28.64 sys 1.525 make -j 5, 271.67 real 204.25 user 28.83 sys 1.546 make -j 6, 263.22 real 203.66 user 29.88 sys 1.596 make -j 7, 260.99 real 204.40 user 29.53 sys 1.609 make -j 8, 261.07 real 204.93 user 29.36 sys 1.609 --(PPro/166, 2 CPUs, 512k internal cache/CPU)---------------------- 2 CPUs Active real/normalized make -j 1, 422.19 real 129.24 user 99.65 sys 1.000 make -j 2, 242.46 real 161.48 user 73.07 sys 1.741 make -j 3, 206.94 real 170.82 user 67.47 sys 2.040 make -j 4, 196.59 real 174.09 user 66.98 sys 2.148 make -j 5, 186.75 real 177.68 user 65.23 sys 2.261 make -j 6, 186.60 real 179.64 user 64.57 sys 2.263 make -j 7, 182.05 real 178.10 user 66.68 sys 2.319 make -j 8, 186.33 real 180.30 user 65.83 sys 2.266 --(Pentium/166, 1 CPU, 512K external cache)------------------------ make -j 1, 556.68 real 299.65 user 24.00 sys make -j 2, 421.50 real 301.13 user 24.48 sys make -j 3, 388.21 real 301.94 user 25.89 sys make -j 4, 376.69 real 302.00 user 26.42 sys make -j 5, 380.96 real 303.30 user 25.82 sys make -j 6, 370.55 real 303.00 user 26.54 sys make -j 7, 369.97 real 303.63 user 27.69 sys make -j 8, 372.43 real 303.41 user 27.85 sys ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Thu Jan 9 10:05:18 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA12931 for smp-outgoing; Thu, 9 Jan 1997 10:05:18 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id KAA12926 for ; Thu, 9 Jan 1997 10:05:13 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id LAA22715; Thu, 9 Jan 1997 11:04:00 -0700 Message-Id: <199701091804.LAA22715@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Chuck Robey cc: FreeBSD-SMP@freebsd.org Subject: Re: Testing new machine In-reply-to: Your message of "Thu, 09 Jan 1997 12:31:49 EST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 09 Jan 1997 11:04:00 -0700 Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hi, > On Thu, 9 Jan 1997, Steve Passe wrote: > > > Hi, > > > > >I suppose I should put the sysctl somewhere in my rc file ... is there any > > >particular place in there (has anything GOT to happen first) ? > > > > I'd suggest /usr/local/etc/rc.d/smp.sh > > Thanks, Steve, but I wasn't looking for a place to declare it as much as > an indication of anything that happens in /etc/rc that HAS to take place > first. I figure I have to hook it in, in either /etc/rc, /etc/rc.i386 or > /etc/rc.local (which could very well call /usr/local/etc/rc.d/smp.sh) but > I don't know if anything in the present startup sequence is required > before turning on multiprocessing. actually I suggested that location because there are some things that need to be done first, but we have NO idea what! See the code marked SMP_AUTOSTART in init_smp.c for what doesn't work. I picked the above location as a point that is at the end of everything, without explaining why... I believe proper form now is to ignore /etc/rc.local, /usr/local/etc/rc.d/xxx.sh scripts are run by /etc/rc. -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Jan 9 10:33:44 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id KAA14246 for smp-outgoing; Thu, 9 Jan 1997 10:33:44 -0800 (PST) Received: from clem.systemsix.com (clem.systemsix.com [198.99.86.131]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id KAA14239 for ; Thu, 9 Jan 1997 10:33:39 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by clem.systemsix.com (8.6.12/8.6.12) with SMTP id LAA22846; Thu, 9 Jan 1997 11:31:47 -0700 Message-Id: <199701091831.LAA22846@clem.systemsix.com> X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol X-Mailer: exmh version 1.6.5 12/11/95 From: Steve Passe To: Chuck Robey cc: FreeBSD-SMP@FreeBSD.org Subject: Re: New stats In-reply-to: Your message of "Thu, 09 Jan 1997 12:52:57 EST." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 09 Jan 1997 11:31:47 -0700 Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk Hi, > --(PPro/166, 2 CPUs, 512k internal cache/CPU)---------------------- > 2 CPUs Active > ... > make -j 8, 186.33 real 180.30 user 65.83 sys 2.266 ^^^^^^^^^^^ I don't understand these times, specifically the user column. On my system I get: time make -j8, 267.32s real 396.20s user 94.68s system ^^^^^^^^^^^^ ie, since 2 CPUs are being used, I get more user time than real time, yet you show slightly less. Are you normalizing that column also? -- Steve Passe | powered by smp@csn.net | FreeBSD From owner-freebsd-smp Thu Jan 9 11:41:25 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id LAA17554 for smp-outgoing; Thu, 9 Jan 1997 11:41:25 -0800 (PST) Received: from algw2.lucent.com (algw2.lucent.com [205.147.213.2]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id LAA17548 for ; Thu, 9 Jan 1997 11:41:16 -0800 (PST) From: ejc@naserver1.cb.lucent.com Received: from nasvr1.cb.lucent.com by alig2.firewall.lucent.com (SMI-8.6/EMS-L sol2) id OAA11347; Thu, 9 Jan 1997 14:40:52 -0500 Received: from barnacle.cb.lucent.com by nasvr1.cb.lucent.com (5.x/EMS-L sol2) id AA02147; Thu, 9 Jan 1997 14:41:31 -0500 Received: by barnacle.cb.lucent.com (SMI-8.6/EMS-1.1 Sol2) id OAA28179; Thu, 9 Jan 1997 14:42:57 -0500 Date: Thu, 9 Jan 1997 14:42:57 -0500 Message-Id: <199701091942.OAA28179@barnacle.cb.lucent.com> To: chuckr@Glue.umd.edu, smp@csn.net Subject: Re: Testing new machine Cc: FreeBSD-SMP@freebsd.org X-Sun-Charset: US-ASCII Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > From nauucp Thu Jan 9 13:41 EST 1997 > Cc: FreeBSD-SMP@freebsd.org > X-Authentication-Warning: clem.systemsix.com: Host localhost didn't use HELO protocol > From: Steve Passe > To: Chuck Robey > Original-Cc: FreeBSD-SMP@freebsd.org > Subject: Re: Testing new machine > Mime-Version: 1.0 > X-Loop: FreeBSD.org > > Hi, > > > On Thu, 9 Jan 1997, Steve Passe wrote: > > > > > Hi, > > > > > > >I suppose I should put the sysctl somewhere in my rc file ... is there any > > > >particular place in there (has anything GOT to happen first) ? > > > > > > I'd suggest /usr/local/etc/rc.d/smp.sh > > > > Thanks, Steve, but I wasn't looking for a place to declare it as much as > > an indication of anything that happens in /etc/rc that HAS to take place > > first. I figure I have to hook it in, in either /etc/rc, /etc/rc.i386 or > > /etc/rc.local (which could very well call /usr/local/etc/rc.d/smp.sh) but > > I don't know if anything in the present startup sequence is required > > before turning on multiprocessing. > > actually I suggested that location because there are some things that need > to be done first, but we have NO idea what! See the code marked SMP_AUTOSTART > in init_smp.c for what doesn't work. I picked the above location as a point > that is at the end of everything, without explaining why... > > I believe proper form now is to ignore /etc/rc.local, > /usr/local/etc/rc.d/xxx.sh scripts are run by /etc/rc. Hi I believe the current favorite place for local config && startup is /usr/local/etc/rc.d/xxx.sh. But that means I would have to clean out my aging /etc/rc.local file and move it to the proper place. So I guess I'm not a believer :-) ejc > > -- > Steve Passe | powered by > smp@csn.net | FreeBSD > > > > From owner-freebsd-smp Thu Jan 9 13:06:15 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id NAA21486 for smp-outgoing; Thu, 9 Jan 1997 13:06:15 -0800 (PST) Received: from po2.glue.umd.edu (root@po2.glue.umd.edu [129.2.128.45]) by freefall.freebsd.org (8.8.4/8.8.4) with ESMTP id NAA21480 for ; Thu, 9 Jan 1997 13:06:13 -0800 (PST) Received: from thurston.eng.umd.edu (thurston.eng.umd.edu [129.2.103.25]) by po2.glue.umd.edu (8.8.3/8.7.3) with ESMTP id QAA19290; Thu, 9 Jan 1997 16:06:10 -0500 (EST) Received: from localhost (chuckr@localhost) by thurston.eng.umd.edu (8.8.3/8.7.3) with SMTP id QAA05910; Thu, 9 Jan 1997 16:06:09 -0500 (EST) X-Authentication-Warning: thurston.eng.umd.edu: chuckr owned process doing -bs Date: Thu, 9 Jan 1997 16:06:09 -0500 (EST) From: Chuck Robey X-Sender: chuckr@thurston.eng.umd.edu To: Steve Passe cc: FreeBSD-SMP@FreeBSD.org Subject: Re: New stats In-Reply-To: <199701091831.LAA22846@clem.systemsix.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-smp@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk On Thu, 9 Jan 1997, Steve Passe wrote: > Hi, > > > --(PPro/166, 2 CPUs, 512k internal cache/CPU)---------------------- > > 2 CPUs Active > > ... > > make -j 8, 186.33 real 180.30 user 65.83 sys 2.266 > ^^^^^^^^^^^ > I don't understand these times, specifically the user column. On my system I > get: > > time make -j8, 267.32s real 396.20s user 94.68s system > ^^^^^^^^^^^^ > > ie, since 2 CPUs are being used, I get more user time than real time, yet > you show slightly less. Are you normalizing that column also? No. the only column I jiggered was the one I added, the one labelled real/normalized, all the rest were directly out of /usr/bin/time. The numbers are repeatable. I double-checked that. > -- > Steve Passe | powered by > smp@csn.net | FreeBSD > > ----------------------------+----------------------------------------------- Chuck Robey | Interests include any kind of voice or data chuckr@eng.umd.edu | communications topic, C programming, and Unix. 9120 Edmonston Ct #302 | Greenbelt, MD 20770 | I run Journey2 and picnic, both FreeBSD (301) 220-2114 | version 3.0 current -- and great FUN! ----------------------------+----------------------------------------------- From owner-freebsd-smp Thu Jan 9 15:53:07 1997 Return-Path: Received: (from root@localhost) by freefall.freebsd.org (8.8.4/8.8.4) id PAA29087 for smp-outgoing; Thu, 9 Jan 1997 15:53:07 -0800 (PST) Received: from phaeton.artisoft.com (phaeton.Artisoft.COM [198.17.250.211]) by freefall.freebsd.org (8.8.4/8.8.4) with SMTP id PAA29082 for ; Thu, 9 Jan 1997 15:53:04 -0800 (PST) Received: (from terry@localhost) by phaeton.artisoft.com (8.6.11/8.6.9) id QAA18917; Thu, 9 Jan 1997 16:42:36 -0700 From: Terry Lambert Message-Id: <199701092342.QAA18917@phaeton.artisoft.com> Subject: Re: unusual question To: phk@critter.dk.tfs.com (Poul-Henning Kamp) Date: Thu, 9 Jan 1997 16:42:36 -0700 (MST) Cc: mef@cs.washington.edu, smp@freebsd.org In-Reply-To: <7514.852822663@critter.dk.tfs.com> from "Poul-Henning Kamp" at Jan 9, 97 04:11:03 pm X-Mailer: ELM [version 2.4 PL24] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-smp@freebsd.org X-Loop: FreeBSD.org Precedence: bulk > Generally, you can allow device-drivers to be multi-threaded only as > far as you have separate pieces of hardware. Ie, in you average > FreeBSD system you could have one thread/cpu per sio port, but only > one for wdc0+wd0+wd1 because they share registers. > > I would expect the "one-lock per device-driver" to be implemented > in the code that calls into the device drivers, simply to avoid > changing all of them, so most device-drivers will probably not > change much if at all. Initially, anyway. Ideally, there would be top/bottom based drivers to implement code for running above and below a HAL; this would ensure maximum concurrency (mostly a problem for SCSI controllers; most other things won't care, but it's the principle of the thing... Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers.