From owner-freebsd-smp Sun Oct 4 01:39:05 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id BAA04324 for freebsd-smp-outgoing; Sun, 4 Oct 1998 01:39:05 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from gilberto.physik.RWTH-Aachen.DE (gilberto.physik.rwth-aachen.de [137.226.30.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id BAA04311 for ; Sun, 4 Oct 1998 01:38:56 -0700 (PDT) (envelope-from kuku@gilberto.physik.RWTH-Aachen.DE) Received: (from kuku@localhost) by gilberto.physik.RWTH-Aachen.DE (8.8.8/8.8.7) id KAA13965 for freebsd-smp@freebsd.org; Sun, 4 Oct 1998 10:38:43 +0200 (MEST) (envelope-from kuku) Date: Sun, 4 Oct 1998 10:38:43 +0200 (MEST) From: Christoph Kukulies Message-Id: <199810040838.KAA13965@gilberto.physik.RWTH-Aachen.DE> To: freebsd-smp@FreeBSD.ORG Subject: hw platform Q - what's a good smp choice these days? Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org With all the new CPU models, higher clock rates, new MBs, I'm wondering what a good solid SMP hardware setup is these days? Would it be for instance a good idea to run a quad Pentium I/233 MMX (ASUS make such a MB afaik)? Or should one look out for used PPros? (Are they still being sold?) Which of the newer intel CPU types 400/450 MHz (and I understand that only intel is the choice for SMP ) is SMP capable? -- Chris Christoph P. U. Kukulies kuku@gil.physik.rwth-aachen.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 4 08:07:20 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA17332 for freebsd-smp-outgoing; Sun, 4 Oct 1998 08:07:20 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from wrath.cs.utah.edu (wrath.cs.utah.edu [155.99.198.100]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA17312 for ; Sun, 4 Oct 1998 08:07:14 -0700 (PDT) (envelope-from vanmaren@fast.cs.utah.edu) Received: from fast.cs.utah.edu (fast.cs.utah.edu [155.99.212.1]) by wrath.cs.utah.edu (8.8.8/8.8.8) with SMTP id JAA21970; Sun, 4 Oct 1998 09:06:40 -0600 (MDT) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id JAA07196; Sun, 4 Oct 1998 09:06:36 -0600 Date: Sun, 4 Oct 1998 09:06:36 -0600 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199810041506.JAA07196@fast.cs.utah.edu> To: freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.RWTH-Aachen.DE Subject: Re: hw platform Q - what's a good smp choice these days? Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I'd go with a dual 350MHz Pentium II. The CPUs are only ~ $315 each! Contrast that to $508 for the 400MHz, or $715 for the 450. You get a 100Mhz bus, which is going to be better than a Pro system. You can get a complete kick-ass system (w/o SCSI, but the IDE driver in -CURRENT does DMA now) for under $2k (plus $1200-$1700 for a nice Sony 21" monitor), with 128MB SDRAM, cdrom, sound, etc. Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 4 09:57:19 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA26792 for freebsd-smp-outgoing; Sun, 4 Oct 1998 09:57:19 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from Ilsa.StevesCafe.com (Ilsa.StevesCafe.com [205.168.119.129]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA26784 for ; Sun, 4 Oct 1998 09:57:17 -0700 (PDT) (envelope-from fbsd@Ilsa.StevesCafe.com) Received: from Ilsa.StevesCafe.com (localhost.StevesCafe.com [127.0.0.1]) by Ilsa.StevesCafe.com (8.8.8/8.8.5) with ESMTP id KAA15784; Sun, 4 Oct 1998 10:56:42 -0600 (MDT) Message-Id: <199810041656.KAA15784@Ilsa.StevesCafe.com> X-Mailer: exmh version 2.0.2 2/24/98 From: Steve Passe To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.RWTH-Aachen.DE Subject: Re: hw platform Q - what's a good smp choice these days? In-reply-to: Your message of "Sun, 04 Oct 1998 09:06:36 MDT." <199810041506.JAA07196@fast.cs.utah.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 04 Oct 1998 10:56:42 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, > Would it be for instance a good idea to run a quad Pentium I/233 MMX ^^^^^^^^^^^^^^^^^ do you mean P5 CPUs? Very bad idea, unless someone gives you one. > Or should one look out for used PPros? (Are they still being sold?) Again, unless you get one for a little above shipping, its a bad purchase choice today. >> I'd go with a dual 350MHz Pentium II. The CPUs are only ~ $315 >> each! Contrast that to $508 for the 400MHz, or $715 for the 450. >> You get a 100Mhz bus, which is going to be better than a Pro system. >> You can get a complete kick-ass system (w/o SCSI, but the IDE >> driver in -CURRENT does DMA now) for under $2k (plus $1200-$1700 >> for a nice Sony 21" monitor), with 128MB SDRAM, cdrom, sound, etc. Don't cheap-out on SCSI, its only about $120 more for u2w capability: ASUS P2B-D Dual Pentium II AGP ATX Motherboard $299 ASUS P2B-DS Pentium II AGP ATX Motherboard $419 Much cheaper than adding a card later... I've really become a fan of the Asus P2B-DS, I can do a "make -j12 buildworld" in about 40 minutes (3 spindles on u2w cheetahs). -- Steve Passe | powered by smp@csn.net | Symmetric MultiProcessor FreeBSD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 4 11:05:21 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA03608 for freebsd-smp-outgoing; Sun, 4 Oct 1998 11:05:21 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from danberlin.resnet.rochester.edu (danberlin.resnet.rochester.edu [128.151.84.217]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA03590 for ; Sun, 4 Oct 1998 11:05:10 -0700 (PDT) (envelope-from root@danberlin.resnet.rochester.edu) Received: from danberlin.resnet.rochester.edu (root@danberlin.resnet.rochester.edu [128.151.84.217]) by danberlin.resnet.rochester.edu (8.9.1a/8.9.1/Debian/GNU) with ESMTP id NAA03304; Sun, 4 Oct 1998 13:59:11 -0400 Date: Sun, 4 Oct 1998 13:59:10 -0400 (EDT) From: root To: Kevin Van Maren cc: freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.RWTH-Aachen.DE Subject: Re: hw platform Q - what's a good smp choice these days? In-Reply-To: <199810041506.JAA07196@fast.cs.utah.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > driver in -CURRENT does DMA now) for under $2k (plus $1200-$1700 > for a nice Sony 21" monitor), with 128MB SDRAM, cdrom, sound, etc. And if you don't mind a refurbed monitor, go to onsale. I just got a CTX EX900 with a 1 year warranty (refurbed by CTX) for 320 bucks. It retails for 800-900. You tend to bid against idiots there (If a monitor has ultra in the name, it'll go for more than one with "EO75" in the name, even though the EO75 is the super high end model, and the ultra is the entry level), so use it to your advantage. --Dan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 4 11:41:06 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA08913 for freebsd-smp-outgoing; Sun, 4 Oct 1998 11:41:06 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from wrath.cs.utah.edu (wrath.cs.utah.edu [155.99.198.100]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA08902 for ; Sun, 4 Oct 1998 11:41:00 -0700 (PDT) (envelope-from vanmaren@fast.cs.utah.edu) Received: from fast.cs.utah.edu (fast.cs.utah.edu [155.99.212.1]) by wrath.cs.utah.edu (8.8.8/8.8.8) with SMTP id MAA24273; Sun, 4 Oct 1998 12:40:32 -0600 (MDT) Received: by fast.cs.utah.edu (8.6.10/utah-2.15-leaf) id MAA08710; Sun, 4 Oct 1998 12:40:32 -0600 Date: Sun, 4 Oct 1998 12:40:32 -0600 From: vanmaren@fast.cs.utah.edu (Kevin Van Maren) Message-Id: <199810041840.MAA08710@fast.cs.utah.edu> To: smp@csn.net Subject: Re: hw platform Q - what's a good smp choice these days? Cc: freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.RWTH-Aachen.DE Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Don't cheap-out on SCSI, its only about $120 more for u2w capability: > > ASUS P2B-D Dual Pentium II AGP ATX Motherboard $299 > ASUS P2B-DS Pentium II AGP ATX Motherboard $419 > > Much cheaper than adding a card later... I've really become a fan of the > Asus P2B-DS, I can do a "make -j12 buildworld" in about 40 minutes (3 > spindles on u2w cheetahs). If you are going to multiple disks and have the money, SCSI is the way to go. But IDE disks are so much less! For a single drive, the new IDE DMA driver does as well as SCSI. I went with a U2W Barracuda and P2B-DS. I can't afford 3 Cheetah drives for my home PC, and some people can't afford $400 extra for SCSI, when a 7200RPM Medalist Pro IDE drive will be basically as fast. But, then again, you bought faster CPUs too. That $400 difference between IDE and SCSI is also the price difference between a single 350MHz-CPU motherboard and a dual-capable MB with a second 350MHz CPU. In servers, or multiple-disk environments, go SCSI. If you have the money, go SCSI. But if you have to cut something for a desk PC, get a dual over SCSI. Kevin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 4 15:15:29 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id PAA07676 for freebsd-smp-outgoing; Sun, 4 Oct 1998 15:15:29 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from silverback.gorilla.net (silverback.gorilla.net [208.128.8.1]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id PAA07623; Sun, 4 Oct 1998 15:14:54 -0700 (PDT) (envelope-from tom@gorilla.net) Received: from peeper.TOJ.org [208.143.84.81] by silverback.gorilla.net with ESMTP (SMTPD32-4.06) id A253323027E; Sun, 04 Oct 1998 17:10:27 CDT Received: (from tom@localhost) by peeper.TOJ.org (8.9.1/8.8.8) id RAA00328; Sun, 4 Oct 1998 17:14:27 -0500 (CDT) (envelope-from tom) Message-ID: <19981004171357.A318@TOJ.org> Date: Sun, 4 Oct 1998 17:13:57 -0500 From: Tom Jackson To: Mike Smith , Tom Jackson Cc: current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: CRUSH after recompile kernel... References: <199810040725.AAA02590@dingo.cdrom.com> <199810040919.CAA03330@dingo.cdrom.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <199810040919.CAA03330@dingo.cdrom.com>; from Mike Smith on Sun, Oct 04, 1998 at 02:19:43AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Okay Mike the new commits worked for the boot loader. To get the elf kernel to work unattended I had to comment out the options DDB and USERCONFIG_BOOT. There's some confusion out there about boot.conf. My /boot.config has the '/boot/loader' in it. I don't even know what the /boot/boot.conf is used for and there should not be a /boot.conf. Will keep fiddling with it, thanks for enabling me to go ELF, all the way. On Sun, Oct 04, 1998 at 02:19:43AM -0700, Mike Smith wrote: > > > > Try other slices, eg: 'ls disk1s1a:'. It's not unlikely that there's a > > > > miscalculation in the slice numbering somewhere. > > > > > > I first wanted to try my Thinkpad with a dangerously dedicated ide and the > > > src and obj nfs mounted from the server (the one that doesn't work). The > > > Thinkpad works fine. > > > > > > The server, asus p2l97-ds w/ 1005 bios, does indeed have another partition > > > before FBSD. 1 is w95, 2 is FBSD boot, and 3 is FBSD. I tried ls'ing every- > > > thing I could think of and nothing works. I'll include the fdisk and disklabel. > > > > > > Any suggestion on what to try? > > > > Hang back and watch for some more commits - I appear to have made some > > mistakes in the 'guess what the root device is' code. On a box I just > > booted with quite old bootblocks, I get the "Can't work out which disk > > we are booting from" message, and the default device is 'disk-1a'. > > Ok. I've committed a few fixes now. I'm still seeing an odd case here > where the information from the previous bootstrap isn't being copied in > correctly, so the default disk is set wrong at startup, but I need to > chase this with Mr Nordier. > > Let me know how you go. > -- Tom -- IMail Server for Windows NT. Evaluation version. Copyright (c) 1995-98 Ipswitch, Inc. http://www.ipswitch.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 4 17:37:08 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id RAA25972 for freebsd-smp-outgoing; Sun, 4 Oct 1998 17:37:08 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from Ilsa.StevesCafe.com (Ilsa.StevesCafe.com [205.168.119.129]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id RAA25961 for ; Sun, 4 Oct 1998 17:37:03 -0700 (PDT) (envelope-from smp@Ilsa.StevesCafe.com) Received: from Ilsa.StevesCafe.com (localhost.StevesCafe.com [127.0.0.1]) by Ilsa.StevesCafe.com (8.8.8/8.8.5) with ESMTP id SAA17049; Sun, 4 Oct 1998 18:36:40 -0600 (MDT) Message-Id: <199810050036.SAA17049@Ilsa.StevesCafe.com> X-Mailer: exmh version 2.0.2 2/24/98 From: Steve Passe To: vanmaren@fast.cs.utah.edu (Kevin Van Maren) cc: freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.RWTH-Aachen.DE Subject: Re: hw platform Q - what's a good smp choice these days? In-reply-to: Your message of "Sun, 04 Oct 1998 12:40:32 MDT." <199810041840.MAA08710@fast.cs.utah.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 04 Oct 1998 18:36:39 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, > > Don't cheap-out on SCSI, its only about $120 more for u2w capability: > > > > ASUS P2B-D Dual Pentium II AGP ATX Motherboard $299 > > ASUS P2B-DS Pentium II AGP ATX Motherboard $419 > > > > Much cheaper than adding a card later... I've really become a fan of the > > Asus P2B-DS, I can do a "make -j12 buildworld" in about 40 minutes (3 > > spindles on u2w cheetahs). > > If you are going to multiple disks and have the money, SCSI is the > way to go. But IDE disks are so much less! For a single drive, > the new IDE DMA driver does as well as SCSI. I agree with this point, but I think of SCSI as much more than a disk controller. I typically hang a jaz or zip drive off it, plus a SCSI DAT tape for backup, or possibly a scanner, or a CD-ROM drive, or ... -- Steve Passe | powered by smp@csn.net | Symmetric MultiProcessor FreeBSD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Oct 4 22:45:36 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA11188 for freebsd-smp-outgoing; Sun, 4 Oct 1998 22:45:36 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from dingo.cdrom.com (castles140.castles.com [208.214.165.140]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA11039; Sun, 4 Oct 1998 22:45:07 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Received: from dingo.cdrom.com (localhost [127.0.0.1]) by dingo.cdrom.com (8.9.1/8.8.8) with ESMTP id WAA00390; Sun, 4 Oct 1998 22:49:47 -0700 (PDT) (envelope-from mike@dingo.cdrom.com) Message-Id: <199810050549.WAA00390@dingo.cdrom.com> X-Mailer: exmh version 2.0.2 2/24/98 To: Tom Jackson cc: Mike Smith , current@FreeBSD.ORG, smp@FreeBSD.ORG Subject: Re: CRUSH after recompile kernel... In-reply-to: Your message of "Sun, 04 Oct 1998 17:13:57 CDT." <19981004171357.A318@TOJ.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 04 Oct 1998 22:49:46 -0700 From: Mike Smith Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Okay Mike the new commits worked for the boot loader. To get the elf kernel > to work unattended I had to comment out the options DDB and USERCONFIG_BOOT. > There's some confusion out there about boot.conf. My /boot.config has the > '/boot/loader' in it. I don't even know what the /boot/boot.conf is used for > and there should not be a /boot.conf. Will keep fiddling with it, thanks for > enabling me to go ELF, all the way. /boot.config is read by the 'initial' bootstrap, the tool which in turn loads /boot/loader. This is transitional; eventually it will be able to go away. /boot/boot.conf (whose name may yet change, eg. /boot/loader.rc - no firm decisions yet) is read as a startup script by the 'loader' program. It can contain commands for this program; you can get a brief summary of these by typing '?' at the loader prompt. More complete help is still on the whiteboard. -- \\ Sometimes you're ahead, \\ Mike Smith \\ sometimes you're behind. \\ mike@smith.net.au \\ The race is long, and in the \\ msmith@freebsd.org \\ end it's only with yourself. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Oct 6 11:35:42 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA13226 for freebsd-smp-outgoing; Tue, 6 Oct 1998 11:35:42 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from post.mail.demon.net (post-12.mail.demon.net [194.217.242.41]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA12819 for ; Tue, 6 Oct 1998 11:33:12 -0700 (PDT) (envelope-from james@westongold.com) Received: from [158.152.96.124] (helo=wgp01.wgold.demon.co.uk) by post.mail.demon.net with esmtp (Exim 2.03 #1) id 0zQbus-0001lZ-00; Tue, 6 Oct 1998 18:32:58 +0000 Received: by WGP01 with Internet Mail Service (5.5.1960.3) id ; Mon, 5 Oct 1998 20:57:24 +0100 Message-ID: <32BABEF63EAED111B2C5204C4F4F502017F7@WGP01> From: James Mansion To: Steve Passe , vanmaren@fast.cs.utah.edu Cc: freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.RWTH-Aachen.DE Subject: RE: hw platform Q - what's a good smp choice these days? Date: Mon, 5 Oct 1998 20:57:22 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.1960.3) Content-Type: text/plain Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Its all very well, but SCSI disks are a LOT more expensive than IDE ones. Might be faster IO per drive with SCSI, but I'd rather have twice the capacity and spend the remainder on more RAM. James > -----Original Message----- > From: Steve Passe [mailto:smp@csn.net] > SNIP > > > If you are going to multiple disks and have the money, SCSI is the > > way to go. But IDE disks are so much less! For a single drive, > > the new IDE DMA driver does as well as SCSI. > > I agree with this point, but I think of SCSI as much more than a disk > controller. I typically hang a jaz or zip drive off it, plus a SCSI > DAT tape for backup, or possibly a scanner, or a CD-ROM drive, or ... > > -- > Steve Passe | powered by > smp@csn.net | Symmetric MultiProcessor FreeBSD > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 06:52:09 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id GAA22289 for freebsd-smp-outgoing; Wed, 7 Oct 1998 06:52:09 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from lorax.ubergeeks.com (lorax.ubergeeks.com [206.205.41.241]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id GAA22217 for ; Wed, 7 Oct 1998 06:52:01 -0700 (PDT) (envelope-from adrian@lorax.ubergeeks.com) Received: from localhost (adrian@localhost) by lorax.ubergeeks.com (8.8.8/8.8.8) with SMTP id JAA04624; Wed, 7 Oct 1998 09:51:19 -0400 (EDT) (envelope-from adrian@lorax.ubergeeks.com) Date: Wed, 7 Oct 1998 09:51:19 -0400 (EDT) From: ADRIAN Filipi-Martin Reply-To: Adrian Filipi-Martin To: James Mansion cc: Steve Passe , vanmaren@fast.cs.utah.edu, freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.rwth-aachen.de Subject: RE: hw platform Q - what's a good smp choice these days? In-Reply-To: <32BABEF63EAED111B2C5204C4F4F502017F7@WGP01> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 5 Oct 1998, James Mansion wrote: > Its all very well, but SCSI disks are a LOT more expensive than IDE > ones. > > Might be faster IO per drive with SCSI, but I'd rather have twice the > capacity and spend the remainder on more RAM. > > James Well, IDE tends to use significantly more CPU for IO than SCSI. Isn't that what does all the DMA work for IDE? If you have cycles to burn, why pay for an SMP configuration? I wouldn't recommend IDE on anything but a single user machine, because of the synchronous access to the drives. (Has the sycnhronous drive access been dropped from EIDE? I don't follow IDE developments much.) Adrian -- [ adrian@ubergeeks.com -- Ubergeeks Consulting -- http://www.ubergeeks.com/ ] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 06:55:47 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id GAA22792 for freebsd-smp-outgoing; Wed, 7 Oct 1998 06:55:47 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from dworkin.amber.org (dworkin.amber.org [209.31.146.74]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id GAA22775 for ; Wed, 7 Oct 1998 06:55:42 -0700 (PDT) (envelope-from petrilli@dworkin.amber.org) Received: (from petrilli@localhost) by dworkin.amber.org (8.9.0/8.9.0) id JAA03206; Wed, 7 Oct 1998 09:55:38 -0400 (EDT) Message-ID: <19981007095538.07811@amber.org> Date: Wed, 7 Oct 1998 09:55:38 -0400 From: "Christopher G. Petrilli" To: freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? Mail-Followup-To: freebsd-smp@FreeBSD.ORG References: <32BABEF63EAED111B2C5204C4F4F502017F7@WGP01> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.89.1i In-Reply-To: <32BABEF63EAED111B2C5204C4F4F502017F7@WGP01>; from James Mansion on Mon, Oct 05, 1998 at 08:57:22PM +0100 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Oct 05, 1998 at 08:57:22PM +0100, James Mansion wrote: > Its all very well, but SCSI disks are a LOT more expensive than IDE > ones. > > Might be faster IO per drive with SCSI, but I'd rather have twice the > capacity and spend the remainder on more RAM. Since nobody asked what this person was gdoing with thier machine, it's difficult to guess as to the best mixture. I will hazard this statement, *IF* the system is not a pure CPU server (i.e. Beowulf, whatever), then it's I/O bound, I promise you :-) I've been building everything from mainframes to Sun Starfires to little embedded machines, and ALWAYS they are I/O bound. When you think you've fixed that they're memory bandwidth bound, they're never CPU bound. Having said that, assuming this machine is aimed at general purpose web/etc, I would recommend getting a dual CPU board, but if you have to cut costs, NEVER cut it in the I/O subsystem on a server (clients are different stories). Also, more spindels is better, and I often try and find a small drive for my root drive, and use another small drive for swap, just to keep it off everything. This is again an I/O at all costs design, and I'm known to put 3 SCSI busses on a machine that has any load. Note that anythign that is heavily disk bound, i.e. database, or news servers, should put even more emphasis on the I/O side of the house---something unfortunately that PCs still are pretty miserable at. Chris > James > > > -----Original Message----- > > From: Steve Passe [mailto:smp@csn.net] > > > SNIP > > > > > If you are going to multiple disks and have the money, SCSI is the > > > way to go. But IDE disks are so much less! For a single drive, > > > the new IDE DMA driver does as well as SCSI. > > > > I agree with this point, but I think of SCSI as much more than a disk > > controller. I typically hang a jaz or zip drive off it, plus a SCSI > > DAT tape for backup, or possibly a scanner, or a CD-ROM drive, or ... > > > > -- > > Steve Passe | powered by > > smp@csn.net | Symmetric MultiProcessor FreeBSD > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-smp" in the body of the message > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message -- | Christopher Petrilli | petrilli@amber.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 08:11:03 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA07422 for freebsd-smp-outgoing; Wed, 7 Oct 1998 08:11:03 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from dworkin.amber.org (dworkin.amber.org [209.31.146.74]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id IAA07413 for ; Wed, 7 Oct 1998 08:10:58 -0700 (PDT) (envelope-from petrilli@dworkin.amber.org) Received: (from petrilli@localhost) by dworkin.amber.org (8.9.0/8.9.0) id LAA03422; Wed, 7 Oct 1998 11:10:54 -0400 (EDT) Message-ID: <19981007111054.39986@amber.org> Date: Wed, 7 Oct 1998 11:10:54 -0400 From: "Christopher G. Petrilli" To: freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? Mail-Followup-To: freebsd-smp@FreeBSD.ORG References: <32BABEF63EAED111B2C5204C4F4F502017F7@WGP01> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.89.1i In-Reply-To: ; from ADRIAN Filipi-Martin on Wed, Oct 07, 1998 at 09:51:19AM -0400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Oct 07, 1998 at 09:51:19AM -0400, ADRIAN Filipi-Martin wrote: > On Mon, 5 Oct 1998, James Mansion wrote: > > > Its all very well, but SCSI disks are a LOT more expensive than IDE > > ones. > > > > Might be faster IO per drive with SCSI, but I'd rather have twice the > > capacity and spend the remainder on more RAM. > > Well, IDE tends to use significantly more CPU for IO than SCSI. > Isn't that what does all the DMA work for IDE? If you have cycles to > burn, why pay for an SMP configuration? I wouldn't recommend IDE on > anything but a single user machine, because of the synchronous access to > the drives. (Has the sycnhronous drive access been dropped from EIDE? I > don't follow IDE developments much.) Well, even under SCSI, most people run it in syncronous mode because it's faster unless your kernel supports full blown tag-command queing, and the drive does reordering, AND you're doing small transfers and high transaction rate. This is kinda the mainframe model of I/O---you send out a bunch of requests to drives for information, then you let the drive respond in whatever order is best given it's current "state".... This is kinda why spindle sync isn't used much that I cna tell, outside of the ultra-high end. The reality is that sync I/O is ok, and not the problem for most people... it's the CPU overhead, as well as the fact that IDE drives tend to be quite slow in general (don't quote BS interface numbers, it comes down to spindle speed and track-track access times). Chris -- | Christopher Petrilli | petrilli@amber.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 10:44:18 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id KAA07155 for freebsd-smp-outgoing; Wed, 7 Oct 1998 10:44:18 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from verdi.nethelp.no (verdi.nethelp.no [158.36.41.162]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id KAA07084 for ; Wed, 7 Oct 1998 10:44:09 -0700 (PDT) (envelope-from sthaug@nethelp.no) From: sthaug@nethelp.no Received: (qmail 23960 invoked by uid 1001); 7 Oct 1998 17:43:49 +0000 (GMT) To: adrian@ubergeeks.com Cc: freebsd-smp@FreeBSD.ORG Subject: RE: hw platform Q - what's a good smp choice these days? In-Reply-To: Your message of "Wed, 7 Oct 1998 09:51:19 -0400 (EDT)" References: X-Mailer: Mew version 1.05+ on Emacs 19.34.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Wed, 07 Oct 1998 19:43:48 +0200 Message-ID: <23958.907782228@verdi.nethelp.no> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Well, IDE tends to use significantly more CPU for IO than SCSI. > Isn't that what does all the DMA work for IDE? Not if you have a reasonably modern PCI chipset and set the correct flags in your kernel config file. >From my own (limited) measurements, there is no significant difference in CPU usage between IDE and SCSI with the wd driver in 3.0. Steinar Haug, Nethelp consulting, sthaug@nethelp.no To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 11:12:14 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA13548 for freebsd-smp-outgoing; Wed, 7 Oct 1998 11:12:14 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from gilberto.physik.RWTH-Aachen.DE (gilberto.physik.rwth-aachen.de [137.226.30.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA13449 for ; Wed, 7 Oct 1998 11:12:01 -0700 (PDT) (envelope-from kuku@gilberto.physik.RWTH-Aachen.DE) Received: (from kuku@localhost) by gilberto.physik.RWTH-Aachen.DE (8.8.8/8.8.7) id UAA00810; Wed, 7 Oct 1998 20:12:02 +0200 (MEST) (envelope-from kuku) Message-ID: <19981007201201.A795@gil.physik.rwth-aachen.de> Date: Wed, 7 Oct 1998 20:12:01 +0200 From: Christoph Kukulies To: Adrian Filipi-Martin , James Mansion Cc: Steve Passe , vanmaren@fast.cs.utah.edu, freebsd-smp@FreeBSD.ORG, kuku@gilberto.physik.RWTH-Aachen.DE Subject: Re: hw platform Q - what's a good smp choice these days? References: <32BABEF63EAED111B2C5204C4F4F502017F7@WGP01> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.91 In-Reply-To: ; from ADRIAN Filipi-Martin on Wed, Oct 07, 1998 at 09:51:19AM -0400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Oct 07, 1998 at 09:51:19AM -0400, ADRIAN Filipi-Martin wrote: > On Mon, 5 Oct 1998, James Mansion wrote: > > > Its all very well, but SCSI disks are a LOT more expensive than IDE > > ones. > > > > Might be faster IO per drive with SCSI, but I'd rather have twice the > > capacity and spend the remainder on more RAM. > > > > James > > Well, IDE tends to use significantly more CPU for IO than SCSI. > Isn't that what does all the DMA work for IDE? If you have cycles to As far as I understand, using Ultra DMA (IDE DMA) doesn't burn cycles since it works like bus master DMA and FreeBSD supports it, doesn't it? > burn, why pay for an SMP configuration? I wouldn't recommend IDE on > anything but a single user machine, because of the synchronous access to > the drives. (Has the sycnhronous drive access been dropped from EIDE? I > don't follow IDE developments much.) > > Adrian > -- > [ adrian@ubergeeks.com -- Ubergeeks Consulting -- http://www.ubergeeks.com/ ] -- --Chris Christoph P. U. Kukulies kuku@gil.physik.rwth-aachen.de To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 11:31:58 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA17831 for freebsd-smp-outgoing; Wed, 7 Oct 1998 11:31:58 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from pinhead.parag.codegen.com (ppp-pm3-02--100.sirius.net [205.134.231.100]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA17810 for ; Wed, 7 Oct 1998 11:31:52 -0700 (PDT) (envelope-from parag@pinhead.parag.codegen.com) Received: from pinhead.parag.codegen.com (localhost.parag.codegen.com [127.0.0.1]) by pinhead.parag.codegen.com (8.9.1/8.8.8) with ESMTP id LAA07841 for ; Wed, 7 Oct 1998 11:26:37 -0700 (PDT) (envelope-from parag@pinhead.parag.codegen.com) Message-Id: <199810071826.LAA07841@pinhead.parag.codegen.com> X-Mailer: exmh version 2.0.2 2/24/98 To: freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? In-reply-to: Your message of "Wed, 07 Oct 1998 11:10:54 EDT." <19981007111054.39986@amber.org> X-Face: =O'Kj74icvU|oS*<7gS/8'\Pbpm}okVj*@UC!IgkmZQAO!W[|iBiMs*|)n*`X ]pW%m>Oz_mK^Gdazsr.Z0/JsFS1uF8gBVIoChGwOy{EK=<6g?aHE`[\S]C]T0Wm X-URL: http://www.codegen.com Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 07 Oct 1998 11:26:37 -0700 From: Parag Patel Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Another point regarding IDE speeds is that the UDMA (33Mb/sec transfer rates) drives can come close to SCSI performance *if* the drivers aren't trying to talk to the PCI-IDE chips in legacy ISA-mode and *if* there is only one master per IDE controller. The BeOS on Intel hardware supports only IDE drives at the moment and has no trouble handling multiple streaming audio and video streams off of a UDMA drive. They were rather pleasantly surprised by the performance. I suspect part of the problem is that FreeBSD's PCI-IDE drivers merely set a few flags and then let the ISA-IDE driver do most of the work in legacy PIO mode. The latest PCI-IDE chips should be entirely programmable using memory-mapped I/O but require all new drivers. One of my back-burner projects is to write an up-to-date driver for PCI-IDE chips to get better performance vs CPU overhead measurements. Another thing to remember with IDE is that you can only talk to one device of a master or slave at a time. To get maximum performance, you should put each IDE device on a separate controller as a master and do not connect anything as a slave on that controller. You definitely don't want to put two hard disks on the same controller. (It's OK to put two relatively slow devices together like CD-ROMs and tape drives which aren't likely to be used simultaneously.) This essentially uses the PCI bus as a connect-disconnect bus to manage multiple transactions to multiple devices, much like the SCSI bus. Another of my projects is to design a PCI plugin board with about 4-6 PCI-IDE chips from CMD and a TI PCI-PCI bridge chip. The idea is to have a whole bunch of IDE plugs to allow having a lot of masters but no slaves. But it's not worth pursuing without a better driver first to see if the performance is acceptable. Heck - it would be the World's Cheapest RAID device using the ccd driver. :-) Of course, I could be completely out to lunch, and I'm sure you folks will tell me. -- Parag To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 13:14:24 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id NAA10487 for freebsd-smp-outgoing; Wed, 7 Oct 1998 13:14:24 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id NAA10368 for ; Wed, 7 Oct 1998 13:14:11 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id NAA23771; Wed, 7 Oct 1998 13:14:02 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp01.primenet.com, id smtpd023754; Wed Oct 7 13:13:59 1998 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id NAA14751; Wed, 7 Oct 1998 13:13:51 -0700 (MST) From: Terry Lambert Message-Id: <199810072013.NAA14751@usr08.primenet.com> Subject: Re: hw platform Q - what's a good smp choice these days? To: petrilli@amber.org (Christopher G. Petrilli) Date: Wed, 7 Oct 1998 20:13:51 +0000 (GMT) Cc: freebsd-smp@FreeBSD.ORG In-Reply-To: <19981007111054.39986@amber.org> from "Christopher G. Petrilli" at Oct 7, 98 11:10:54 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > Its all very well, but SCSI disks are a LOT more expensive than IDE > > > ones. No, they aren't. > > > Might be faster IO per drive with SCSI, but I'd rather have twice the > > > capacity and spend the remainder on more RAM. The I/O is faster by a factor of N, where N is the number of tagged commands supported by the drive. For IBM drives, SCSI can be as much as 64 times faster than IDE because it can be as much as 64 times more concurrent. > > Well, IDE tends to use significantly more CPU for IO than SCSI. False. This applies only to PIO based IDE controllers. The controllers that support DMA (or "Ultra DMA") can transfer with similar load to their SCSI counterparts. That's burst rate. For sustained rate, concurrency is an issue; see abbove for "tagged command queueing". > > Isn't that what does all the DMA work for IDE? Nope. See above. Either you use PIO, where the transfers are via CPU copies, or you use DMA, where the transfers are via bus mastering, and the CPU is idle during the transfer. > > If you have cycles to burn, why pay for an SMP configuration? Good question, but not relevent in this context, since DMA IDE doesn't burn cycles (only PIO DMA). > > I wouldn't recommend IDE on anything but a single user machine, > > because of the synchronous access to the drives. (Has the > > sycnhronous drive access been dropped from EIDE? I don't follow > > IDE developments much.) EIDE supports tagged command queuing. EIDE drives do not support tagged commands, however, so it's rather irrelevent. > Well, even under SCSI, most people run it in syncronous mode because This is a different meaning of "synchronous". Tagged commands aren't asynchronous, they're concurrent. Probably the source of the confusion. > it's faster unless your kernel supports full blown tag-command queing, > and the drive does reordering, AND you're doing small transfers and Drive reordering is bad for cached writes, and OK for tagged commands, so long as the data is committed to stable storage in the correct order. > high transaction rate. This is kinda the mainframe model of I/O---you > send out a bunch of requests to drives for information, then you let > the drive respond in whatever order is best given it's current > "state".... Yes. And the High end PC server model, as well. > This is kinda why spindle sync isn't used much that I cna tell, outside > of the ultra-high end. Actually, Rod Grimes distributed patches for spindle sync to be used with CCD on FreeBSD about two years ago. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 14:01:02 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA21762 for freebsd-smp-outgoing; Wed, 7 Oct 1998 14:01:02 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from post.mail.demon.net (post-12.mail.demon.net [194.217.242.41]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA21699 for ; Wed, 7 Oct 1998 14:00:28 -0700 (PDT) (envelope-from james@westongold.com) Received: from [158.152.96.124] (helo=wgp01.wgold.demon.co.uk) by post.mail.demon.net with esmtp (Exim 2.03 #1) id 0zR0gl-0001KC-00; Wed, 7 Oct 1998 21:00:04 +0000 Received: by WGP01 with Internet Mail Service (5.5.1960.3) id ; Wed, 7 Oct 1998 21:25:38 +0100 Message-ID: <32BABEF63EAED111B2C5204C4F4F502017F9@WGP01> From: James Mansion To: "Christopher G. Petrilli" , freebsd-smp@FreeBSD.ORG Subject: RE: hw platform Q - what's a good smp choice these days? Date: Wed, 7 Oct 1998 21:25:23 +0100 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.1960.3) Content-Type: text/plain; charset="iso-8859-1" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Well ... I'm going to disagree with you. At least to some extent. It is undeniable that SCSI is (at least in theory) a great deal more sophisticated and that an IDE solution (even a UDMA one) will tend to eat more CPU, and have more queueing of IO requests in the system. I'm not going to argue that SCSI isn't better in an absolute sense. But some people are dramatically oversimplifying the issue, and 'opposition' to IDE and snide remarks are as useless (and as childish and stupid) as similar comments about Microsoft. However, > -----Original Message----- > From: Christopher G. Petrilli [mailto:petrilli@amber.org] > difficult to guess as to the best mixture. I will hazard this > statement, *IF* the system is not a pure CPU server (i.e. Beowulf, > whatever), then it's I/O bound, I promise you :-) I've been building No. Absolutely no. I promise you ;-) > everything from mainframes to Sun Starfires to little embedded > machines, and ALWAYS they are I/O bound. When you think you've fixed No they aren't. Maybe your experience is that they are, but mine is not. It may be that I am spoilt having worked primarily in investment banks. (Lets be clear, I am objecting to your use of 'ALWAYS') My observation has been that systems that have been sensibly configured are net IO bound or CPU bound, but rarely disk transfer bound. Sometimes seek bound, I grant. My experience is mostly with trading systems for derivatives. These tend to behave more like decision support systems than transaction processing systems. On the Sybase systems I'm most familiar with, CPU becomes the limiting factor quickly - even on mullti-CPU 'big iron' from Sun and HP. Where this hasn't been the case, some minor tuning to avoid table scans and help index coverage has been enough to bring the working set well within affordable RAM. On web servers, it is usually the case that you are either bound on your ISP connection, or are CPU bound on CGI/ISAPI/whatever running dynamic services. FTP and some file serving I guess can have very large working sets and be IO bound - fine. None of the systems I've worked with have been very dependent on non-transactional disk stores. From what I can tell, the sensitivity of news servers to disk performance is because news servers tend to use naff software that works like a file server rather than a DBMS. And hopefully the soft writes will dramatically improve things if the memory is there for buffering and the traffic is bursty. If you're running a turnkey system with a bunch of users I'd still think very carefully before going with the knee-jerk reaction that its not a single user client and should be SCSI. In this country a competitive price for a fast-wde 9GB 7200RPC SCSI drive is about UKP350. An 8GB UDMA IDE is less than half that. The UKP200 difference will buy you 192MB of memory, and the controller is probably right there too. UKP200 gets you a 330MHz CPU if you have the motherboard for it, as an alternative. If the 192MB is the difference between swapping and running from RAM with a good deal of file system cached, its the smart way to go. There's no simple answer to which is better. A definitive 'servers need SCSI' is a gross oversimplification - if you're likely to be seek bound then you would be well to see how many bus-master IDE controllers you can handle, and use multiple 4GB or 8GB UDMA IDE spindles - they're a steal. My current and previous machines - apart from the portable I'm using to write this - have been SCSI. Its served me well as a platform for handling decent CD-ROM and DAT as well as disk. But running out of disk space has always been more of a headache than waiting an extra couple of seconds when the system is busy, and I don't know many fileservers that don't suffer from this. > that they're memory bandwidth bound, they're never CPU bound. > > Having said that, assuming this machine is aimed at general purpose > web/etc, I would recommend getting a dual CPU board, but if > you have to > cut costs, NEVER cut it in the I/O subsystem on a server (clients are > different stories). Also, more spindels is better, and I > often try and > find a small drive for my root drive, and use another small drive for > swap, just to keep it off everything. This is again an I/O at all > costs design, and I'm known to put 3 SCSI busses on a machine that has > any load. I agree that cutting seeks by splitting IOs is a good idea, and any database tuning book will tell you that. But for any budget, you can afford way more IDEs and if you're performance bound on your swap device then you really have screwed up the way the system budget was spent. > > Note that anythign that is heavily disk bound, i.e. database, or news > servers, should put even more emphasis on the I/O side of the > house---something unfortunately that PCs still are pretty > miserable at. Not sure this is a wise thing to say. FreeBSD does rather well at controlling IO and you have a hard time getting mid-range Suns and HPs to hit disks as hard as a PC will, simply because its so much easier to get 80MB/s subsystems and disks for your PC. > > Chris > > > James > > > > > -----Original Message----- > > > From: Steve Passe [mailto:smp@csn.net] > > > > > SNIP > > > > > > > If you are going to multiple disks and have the money, > SCSI is the > > > > way to go. But IDE disks are so much less! For a single drive, > > > > the new IDE DMA driver does as well as SCSI. > > > > > > I agree with this point, but I think of SCSI as much more > than a disk > > > controller. I typically hang a jaz or zip drive off it, > plus a SCSI > > > DAT tape for backup, or possibly a scanner, or a CD-ROM > drive, or ... > > > > > > -- > > > Steve Passe | powered by > > > smp@csn.net | Symmetric MultiProcessor FreeBSD > > > > > > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > > with "unsubscribe freebsd-smp" in the body of the message > > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-smp" in the body of the message > > -- > | Christopher Petrilli > | petrilli@amber.org > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 14:12:01 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA23623 for freebsd-smp-outgoing; Wed, 7 Oct 1998 14:12:01 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from math.berkeley.edu (math.Berkeley.EDU [128.32.183.94]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA23592 for ; Wed, 7 Oct 1998 14:11:49 -0700 (PDT) (envelope-from dan@math.berkeley.edu) Received: (from dan@localhost) by math.berkeley.edu (8.8.7/8.8.7) id OAA00662; Wed, 7 Oct 1998 14:11:25 -0700 (PDT) Date: Wed, 7 Oct 1998 14:11:25 -0700 (PDT) From: dan@math.berkeley.edu (Dan Strick) Message-Id: <199810072111.OAA00662@math.berkeley.edu> To: tlambert@primenet.com Subject: Re: hw platform Q - what's a good smp choice these days? Cc: dan@math.berkeley.edu, freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > > Might be faster IO per drive with SCSI, but I'd rather have twice the > > > > capacity and spend the remainder on more RAM. > > The I/O is faster by a factor of N, where N is the number of tagged > commands supported by the drive. For IBM drives, SCSI can be as much > as 64 times faster than IDE because it can be as much as 64 times more > concurrent. Did I ever buy a used car from you? Dan Strick dan@math.berkeley.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 14:50:53 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA00562 for freebsd-smp-outgoing; Wed, 7 Oct 1998 14:50:53 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from dworkin.amber.org (dworkin.amber.org [209.31.146.74]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA00539 for ; Wed, 7 Oct 1998 14:50:44 -0700 (PDT) (envelope-from petrilli@dworkin.amber.org) Received: (from petrilli@localhost) by dworkin.amber.org (8.9.0/8.9.0) id RAA24355; Wed, 7 Oct 1998 17:50:38 -0400 (EDT) Message-ID: <19981007175038.58996@amber.org> Date: Wed, 7 Oct 1998 17:50:38 -0400 From: "Christopher G. Petrilli" To: freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? Mail-Followup-To: freebsd-smp@FreeBSD.ORG References: <32BABEF63EAED111B2C5204C4F4F502017F9@WGP01> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.89.1i In-Reply-To: <32BABEF63EAED111B2C5204C4F4F502017F9@WGP01>; from James Mansion on Wed, Oct 07, 1998 at 09:25:23PM +0100 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Oct 07, 1998 at 09:25:23PM +0100, James Mansion wrote: > > everything from mainframes to Sun Starfires to little embedded > > machines, and ALWAYS they are I/O bound. When you think you've fixed > > No they aren't. Well, I guess I misphrased that, in that in tuning you always start with the I/O side, not the CPU side, as most cases heavily loaded systems respond slowly not because of CPU problems, but because of unpredictable I/O subsystems. Hense, this is why large systems (mini/mainframe) use large quantities of processors to handle the I/O side and often drives are heavily multi-ported (something that gets really hairy). > Maybe your experience is that they are, but mine is not. It may be that > I am spoilt having worked primarily in investment banks. (Lets be > clear, > I am objecting to your use of 'ALWAYS') > > My observation has been that systems that have been sensibly > configured are net IO bound or CPU bound, but rarely disk transfer > bound. Sometimes seek bound, I grant. Net I/O bound is I/O bound, it's not always disk. But "sensibly" configured describes very few systems, and therefore this discussion. Most people are obsessed with megahertz, and confuse performance inside the cache and throughput throughout the entire system. I/O can also be memory bandwidth----which begs the question of why banking isn't more popular in PCs. > My experience is mostly with trading systems for derivatives. These > tend to behave more like decision support systems than transaction > processing systems. On the Sybase systems I'm most familiar with, > CPU becomes the limiting factor quickly - even on mullti-CPU > 'big iron' from Sun and HP. Where this hasn't been the case, some > minor tuning to avoid table scans and help index coverage has been > enough > to bring the working set well within affordable RAM. I suspect these look nothing like PCs with IDe drives though :-) The point is that MOST servers are not configured correctly. The slow movement int he PC industry to I2O is promissing, but it'd be nice if you could offload the entire file system to a processor---but that might be dreaming at this point. The CPU shouldn't be involved in the calculation of i-nodes, it's irrelevnt to a CPU's job. > On web servers, it is usually the case that you are either bound on your > ISP connection, or are CPU bound on CGI/ISAPI/whatever running dynamic > services. Alas, web servers are a weird situation ebcause the Internet as a whole is so obscenely slow in the grand scheme of things. > I agree that cutting seeks by splitting IOs is a good idea, and any > database > tuning book will tell you that. But for any budget, you can afford way > more > IDEs and if you're performance bound on your swap device then you really > have screwed up the way the system budget was spent. Once you start adding large numbers of drives though the cost effectiveness goes away on IDE... you can only practically handle 1 drive per controller (slave drives are a kludge and performance limiting), and you start having to add controllers into the machine. Given there are currently no "massive" controllers for IDE, you MIGHT be able to control 4x2 + 2 (4 PCI slots) drives, which is a rather small system in many cases. This could be contorlled by a single SCSI controller and probably provide roughly the same bandwidth, and substantially less hassle, without sucking up all the slots. > > Note that anythign that is heavily disk bound, i.e. database, or news > > servers, should put even more emphasis on the I/O side of the > > house---something unfortunately that PCs still are pretty > > miserable at. > > Not sure this is a wise thing to say. FreeBSD does rather well at > controlling > IO and you have a hard time getting mid-range Suns and HPs to hit disks > as hard > as a PC will, simply because its so much easier to get 80MB/s subsystems > and > disks for your PC. Specs and reality are very different ideas :-) Most PCs only have 1 PCI bus, and it's trivial to saturate. Most PCs have dinky memory subsystems (do the calculations of how fast a PII/450 is v. SDRAM's bandwidth). Don't get me wrong, I think FreeBSD is fabulous for 99% of the world, especially if some thought is given to offloading RAID, etc to hardware, and I use it almost exclusively for my desktop and all mid/small projects. But I wouldn't look at it for creating huge disk farms, you can't get PC hardware that works that way---you can get INtel hardware that will, see DG and Sequent. Chris NOW, FreeBSD on a Sequent NUMAQ that's interesting :-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 18:02:21 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA10602 for freebsd-smp-outgoing; Wed, 7 Oct 1998 18:02:21 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA10584 for ; Wed, 7 Oct 1998 18:02:14 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.8.8/8.8.8) id SAA15572; Wed, 7 Oct 1998 18:02:08 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp01.primenet.com, id smtpd015540; Wed Oct 7 18:02:01 1998 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id SAA27130; Wed, 7 Oct 1998 18:01:58 -0700 (MST) From: Terry Lambert Message-Id: <199810080101.SAA27130@usr08.primenet.com> Subject: Re: hw platform Q - what's a good smp choice these days? To: dan@math.berkeley.edu (Dan Strick) Date: Thu, 8 Oct 1998 01:01:56 +0000 (GMT) Cc: tlambert@primenet.com, dan@math.berkeley.edu, freebsd-smp@FreeBSD.ORG In-Reply-To: <199810072111.OAA00662@math.berkeley.edu> from "Dan Strick" at Oct 7, 98 02:11:25 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > > > Might be faster IO per drive with SCSI, but I'd rather have twice the > > > > > capacity and spend the remainder on more RAM. > > > > The I/O is faster by a factor of N, where N is the number of tagged > > commands supported by the drive. For IBM drives, SCSI can be as much > > as 64 times faster than IDE because it can be as much as 64 times more > > concurrent. > > Did I ever buy a used car from you? N times the driver-controller-drive + drive-controller-drive latency. TCP/IP sliding windows work the same way: instead of N latencies for N packets, you get 1 latency amortized across N packets. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 18:15:05 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA12385 for freebsd-smp-outgoing; Wed, 7 Oct 1998 18:15:05 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA11860 for ; Wed, 7 Oct 1998 18:11:37 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.8.8/8.8.8) id SAA25569; Wed, 7 Oct 1998 18:10:47 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp04.primenet.com, id smtpd025523; Wed Oct 7 18:10:46 1998 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id SAA27629; Wed, 7 Oct 1998 18:10:44 -0700 (MST) From: Terry Lambert Message-Id: <199810080110.SAA27629@usr08.primenet.com> Subject: Re: hw platform Q - what's a good smp choice these days? To: parag@cgt.com (Parag Patel) Date: Thu, 8 Oct 1998 01:10:44 +0000 (GMT) Cc: freebsd-smp@FreeBSD.ORG In-Reply-To: <199810071826.LAA07841@pinhead.parag.codegen.com> from "Parag Patel" at Oct 7, 98 11:26:37 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > The BeOS on Intel hardware supports only IDE drives at the moment and > has no trouble handling multiple streaming audio and video streams off > of a UDMA drive. They were rather pleasantly surprised by the performance. You know, you're the second person who has told me that my BeOS preview for Intel can't possibly be running of my JAZ drive and my NCR SCSI controller. Just because it says it's not supported doesn't mean it won't work; you just can install directly to a SCSI machine with the boot floppy. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 18:55:15 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA18495 for freebsd-smp-outgoing; Wed, 7 Oct 1998 18:55:15 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from math.berkeley.edu (math.Berkeley.EDU [128.32.183.94]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA18490 for ; Wed, 7 Oct 1998 18:55:14 -0700 (PDT) (envelope-from dan@math.berkeley.edu) Received: (from dan@localhost) by math.berkeley.edu (8.8.7/8.8.7) id SAA16965; Wed, 7 Oct 1998 18:55:08 -0700 (PDT) Date: Wed, 7 Oct 1998 18:55:08 -0700 (PDT) From: dan@math.berkeley.edu (Dan Strick) Message-Id: <199810080155.SAA16965@math.berkeley.edu> To: tlambert@primenet.com Subject: Re: hw platform Q - what's a good smp choice these days? Cc: dan@math.berkeley.edu, freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > N times the driver-controller-drive + drive-controller-drive latency. > > TCP/IP sliding windows work the same way: instead of N latencies > for N packets, you get 1 latency amortized across N packets. Unless the i/o commands are for contiguous sectors, per command latencies are usually much less than head/disk motion latencies, so you normally don't get to effectively eliminate per command latencies by overlapping them. I don't believe a drive supporting 64 simultaneous tagged commands will execute normal I/O patterns anywhere near 64 times as fast as a drive that only executes one command at a time. I am prepared to believe perhaps twice the performance on a relatively large I/O load (and not even that if the disk driver is "smart" about I/O reordering and scatter/gather dma). Have you got actual real-life performance measurements? Dan Strick dan@math.berkeley.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 20:09:59 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id UAA00410 for freebsd-smp-outgoing; Wed, 7 Oct 1998 20:09:59 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from danberlin.resnet.rochester.edu (danberlin.resnet.rochester.edu [128.151.84.217]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id UAA00384 for ; Wed, 7 Oct 1998 20:09:47 -0700 (PDT) (envelope-from root@danberlin.resnet.rochester.edu) Received: from danberlin.resnet.rochester.edu (root@danberlin.resnet.rochester.edu [128.151.84.217]) by danberlin.resnet.rochester.edu (8.9.1a/8.9.1/Debian/GNU) with ESMTP id XAA11638; Wed, 7 Oct 1998 23:03:18 -0400 Date: Wed, 7 Oct 1998 23:03:17 -0400 (EDT) From: Daniel Berlin To: Terry Lambert cc: Parag Patel , freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? In-Reply-To: <199810080110.SAA27629@usr08.primenet.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Only if you have a special kernel/the drivers (i have them handy). The SCSI support was never put out (except buslogic) because of a CHS bug in DriveSetup that causes IDE partitions to be trashed. --Dan > > You know, you're the second person who has told me that my BeOS > preview for Intel can't possibly be running of my JAZ drive and > my NCR SCSI controller. > > Just because it says it's not supported doesn't mean it won't work; > you just can install directly to a SCSI machine with the boot floppy. > > > Terry Lambert > terry@lambert.org > --- > Any opinions in this posting are my own and not those of my present > or previous employers. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Oct 7 23:15:09 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id XAA29913 for freebsd-smp-outgoing; Wed, 7 Oct 1998 23:15:09 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from verdi.nethelp.no (verdi.nethelp.no [158.36.41.162]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id XAA29899 for ; Wed, 7 Oct 1998 23:15:04 -0700 (PDT) (envelope-from sthaug@nethelp.no) From: sthaug@nethelp.no Received: (qmail 28897 invoked by uid 1001); 8 Oct 1998 06:14:57 +0000 (GMT) To: tlambert@primenet.com Cc: petrilli@amber.org, freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? In-Reply-To: Your message of "Wed, 7 Oct 1998 20:13:51 +0000 (GMT)" References: <199810072013.NAA14751@usr08.primenet.com> X-Mailer: Mew version 1.05+ on Emacs 19.34.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Thu, 08 Oct 1998 08:14:57 +0200 Message-ID: <28895.907827297@verdi.nethelp.no> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > > > Its all very well, but SCSI disks are a LOT more expensive than IDE > > > > ones. > > No, they aren't. Well, it depends on what you mean by "a lot". Here on Norway, SCSI disks are 50% - 100% more expensive than IDE of the same rotational speed and capacity. This difference is large enough to be significant. > That's burst rate. For sustained rate, concurrency is an issue; see > abbove for "tagged command queueing". For sustained rate (sequential read, one process), I can read data faster off an IBM 16 GB 5400 RPM drive than a 9 GB 7200 RPM Barracuda (this is not the LP model). This is of course just one data point, but I've seen the same in a number of other comparisons using sequential reads. Steinar Haug, Nethelp consulting, sthaug@nethelp.no To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 8 05:17:19 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id FAA21039 for freebsd-smp-outgoing; Thu, 8 Oct 1998 05:17:19 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from lexmark.lexmark.com (interlock2.lexmark.com [192.146.101.10]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id FAA21034 for ; Thu, 8 Oct 1998 05:17:17 -0700 (PDT) (envelope-from fireston@lexmark.com) Received: by interlock2.lexmark.com id AB06450 (InterLock SMTP Gateway 3.0 for freebsd-smp@FreeBSD.ORG); Thu, 8 Oct 1998 08:17:11 -0400 Message-Id: <199810081217.AB06450@interlock2.lexmark.com> Received: by interlock2.lexmark.com (Protected-side Proxy Mail Agent-1); Thu, 8 Oct 1998 08:17:11 -0400 From: Mik Firestone Subject: subscribe To: freebsd-smp@FreeBSD.ORG Date: Thu, 8 Oct 1998 08:16:49 +0000 (GMT) Reply-To: fireston@lexmark.com X-Mailer: ELM [version 2.4ME+ PL43 (25)] Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org subscribe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 8 09:46:35 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id JAA00650 for freebsd-smp-outgoing; Thu, 8 Oct 1998 09:46:35 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from sasami.jurai.net (sasami.jurai.net [207.153.65.3]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id JAA00633 for ; Thu, 8 Oct 1998 09:46:29 -0700 (PDT) (envelope-from winter@jurai.net) Received: from localhost (winter@localhost) by sasami.jurai.net (8.8.8/8.8.7) with SMTP id MAA05365; Thu, 8 Oct 1998 12:45:31 -0400 (EDT) Date: Thu, 8 Oct 1998 12:45:31 -0400 (EDT) From: "Matthew N. Dodd" To: sthaug@nethelp.no cc: tlambert@primenet.com, petrilli@amber.org, freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? In-Reply-To: <28895.907827297@verdi.nethelp.no> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 8 Oct 1998 sthaug@nethelp.no wrote: > For sustained rate (sequential read, one process), I can read data faster > off an IBM 16 GB 5400 RPM drive than a 9 GB 7200 RPM Barracuda (this is > not the LP model). This is of course just one data point, but I've seen > the same in a number of other comparisons using sequential reads. Umm... How is it surprising that a disk with a higher data density has faster sustained transfer rate? Consider that the drive has nearly two times as many bits per inch and travels at 75% of the speed of the other drive. Grab a 4gig Quantum Bigfoot and read the outermost track and see what kind of insane sustained rate you get off of that. :) Besides, your IBM is fairly new while the Seagate is at least 3 or 4 years old. -- | Matthew N. Dodd | 78 280Z | 75 164E | 84 245DL | FreeBSD/NetBSD/Sprite/VMS | | winter@jurai.net | This Space For Rent | ix86,sparc,m68k,pmax,vax | | http://www.jurai.net/~winter | Are you k-rad elite enough for my webpage? | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 8 11:20:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA17926 for freebsd-smp-outgoing; Thu, 8 Oct 1998 11:20:34 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA17828 for ; Thu, 8 Oct 1998 11:20:14 -0700 (PDT) (envelope-from tlambert@usr06.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.8.8/8.8.8) id LAA06968; Thu, 8 Oct 1998 11:20:06 -0700 (MST) Received: from usr06.primenet.com(206.165.6.206) via SMTP by smtp02.primenet.com, id smtpd006888; Thu Oct 8 11:19:57 1998 Received: (from tlambert@localhost) by usr06.primenet.com (8.8.5/8.8.5) id LAA04334; Thu, 8 Oct 1998 11:19:55 -0700 (MST) From: Terry Lambert Message-Id: <199810081819.LAA04334@usr06.primenet.com> Subject: Re: hw platform Q - what's a good smp choice these days? To: dan@math.berkeley.edu (Dan Strick) Date: Thu, 8 Oct 1998 18:19:55 +0000 (GMT) Cc: tlambert@primenet.com, dan@math.berkeley.edu, freebsd-smp@FreeBSD.ORG In-Reply-To: <199810080155.SAA16965@math.berkeley.edu> from "Dan Strick" at Oct 7, 98 06:55:08 pm X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > N times the driver-controller-drive + drive-controller-drive latency. > > > > TCP/IP sliding windows work the same way: instead of N latencies > > for N packets, you get 1 latency amortized across N packets. > > Unless the i/o commands are for contiguous sectors, per command > latencies are usually much less than head/disk motion latencies, > so you normally don't get to effectively eliminate per command > latencies by overlapping them. I don't believe a drive supporting > 64 simultaneous tagged commands will execute normal I/O patterns > anywhere near 64 times as fast as a drive that only executes > one command at a time. I am prepared to believe perhaps twice > the performance on a relatively large I/O load (and not even > that if the disk driver is "smart" about I/O reordering and > scatter/gather dma). Have you got actual real-life performance > measurements? 64 times faster was 64 times less latency. I thought I had clarified that, but apparently not. Yes, I realize that I didn't choose my words with sufficient care. I am striving to do so in this message, so of course, it's taking me about 8 times longer to compose it. Yes, I expect a *correctly functioning drive* to reorder tagged command requests which have been enqueued to it, but not yet completed. This was the basis of the discussion of the SCSI write caching facility: that write caches tend to reorder requests without notifying the controller, whereas tagged requests are identified on completion by the associated tag. As a result, a system that enforces order in software in the host OS (such as soft updates) can *know* that the information has been committed to stable storage before it attempts a transaction that depends on the previous transaction having been committed to stable storage. The idea behind write caching is that the drive lies, and says it has done something that it has not. This is not necessarily a bad thing, if the drive does not reorder requests independent of the tags used to enqueue them. Justin claims that there exist drives that obey this semantic. In any case, it should be obvious that commands to a SCSI drive sent as: [ ] [ ] [ ] [ ] [ ] [ ] Will complete before commands sent to an IDE drive as: [ ][ ][ ][ ][ ][ ] Because: [ ] is smaller than: [ ] The only remaining issue is how much overlap does latency through a vastly slower (than the CPU clock) memory and I/O bus introduces; I am willing to give you that it may be a smaller overlap, i.e.: [ ] [ ] But on a loaded system (i.e., more than one process -- hence the useless nature of microbenchmarks), *any* overlap is amplified. You should also note that the latency may, in fact, need to be propagated all the way to the application layer(*), and with a 10ms quantum, this effect can be large (if this were not true, then programs such as "team" and "ddd", as well as the entirety of the POSIX async I/O subsystem would be rather fruitless pursuits). (*) because an application may need to make transactional guarantees to imply state between, for example, a database record file and a database index file. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 8 11:35:37 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA20442 for freebsd-smp-outgoing; Thu, 8 Oct 1998 11:35:37 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from verdi.nethelp.no (verdi.nethelp.no [158.36.41.162]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id LAA20428 for ; Thu, 8 Oct 1998 11:35:28 -0700 (PDT) (envelope-from sthaug@nethelp.no) From: sthaug@nethelp.no Received: (qmail 5304 invoked by uid 1001); 8 Oct 1998 18:35:16 +0000 (GMT) To: winter@jurai.net Cc: freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? In-Reply-To: Your message of "Thu, 8 Oct 1998 12:45:31 -0400 (EDT)" References: X-Mailer: Mew version 1.05+ on Emacs 19.34.2 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Date: Thu, 08 Oct 1998 20:35:16 +0200 Message-ID: <5302.907871716@verdi.nethelp.no> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > For sustained rate (sequential read, one process), I can read data faster > > off an IBM 16 GB 5400 RPM drive than a 9 GB 7200 RPM Barracuda (this is > > not the LP model). This is of course just one data point, but I've seen > > the same in a number of other comparisons using sequential reads. > > Umm... How is it surprising that a disk with a higher data density has > faster sustained transfer rate? Consider that the drive has nearly two > times as many bits per inch and travels at 75% of the speed of the other > drive. It's not surprising to me. It may be surprising to those who believe that SCSI is always faster. > Besides, your IBM is fairly new while the Seagate is at least 3 or 4 years > old. The Seagate was bought last fall. The Norwegian vendor couldn't deliver the 9 GB Barracuda LP then. So I don't think the comparison is all that unfair, actually. Steinar Haug, Nethelp consulting, sthaug@nethelp.no To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Oct 8 11:44:33 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA21389 for freebsd-smp-outgoing; Thu, 8 Oct 1998 11:44:33 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from sasami.jurai.net (sasami.jurai.net [207.153.65.3]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id LAA21369 for ; Thu, 8 Oct 1998 11:44:17 -0700 (PDT) (envelope-from winter@jurai.net) Received: from localhost (winter@localhost) by sasami.jurai.net (8.8.8/8.8.7) with SMTP id OAA06818; Thu, 8 Oct 1998 14:44:00 -0400 (EDT) Date: Thu, 8 Oct 1998 14:44:00 -0400 (EDT) From: "Matthew N. Dodd" To: sthaug@nethelp.no cc: freebsd-smp@FreeBSD.ORG Subject: Re: hw platform Q - what's a good smp choice these days? In-Reply-To: <5302.907871716@verdi.nethelp.no> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 8 Oct 1998 sthaug@nethelp.no wrote: > It's not surprising to me. It may be surprising to those who believe > that SCSI is always faster. You missed it again, this isn't a SCSI vs. IDE comparison at all. Get a 9 gig and a 16 gig IDE and you'll find the 16 gig is faster. SCSI will be faster when a higher degree of concurrency is needed. While the 16 gig drive may have an easier time on seeks due to it's higher data density, the interface protocol (IDE) will not be sufficiently concurrent to maintain it's speed over the 'slower' SCSI drive. -- | Matthew N. Dodd | 78 280Z | 75 164E | 84 245DL | FreeBSD/NetBSD/Sprite/VMS | | winter@jurai.net | This Space For Rent | ix86,sparc,m68k,pmax,vax | | http://www.jurai.net/~winter | Are you k-rad elite enough for my webpage? | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 9 00:14:21 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id AAA11938 for freebsd-smp-outgoing; Fri, 9 Oct 1998 00:14:21 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from math.berkeley.edu (math.Berkeley.EDU [128.32.183.94]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id AAA11933 for ; Fri, 9 Oct 1998 00:14:20 -0700 (PDT) (envelope-from dan@math.berkeley.edu) Received: (from dan@localhost) by math.berkeley.edu (8.8.7/8.8.7) id AAA02765; Fri, 9 Oct 1998 00:14:12 -0700 (PDT) Date: Fri, 9 Oct 1998 00:14:12 -0700 (PDT) From: dan@math.berkeley.edu (Dan Strick) Message-Id: <199810090714.AAA02765@math.berkeley.edu> To: tlambert@primenet.com Subject: Re: hw platform Q - what's a good smp choice these days? Cc: dan@math.berkeley.edu, freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > In any case, it should be obvious that commands to a SCSI drive > sent as: > > [ ] > [ ] > [ ] > [ ] > [ ] > [ ] > > Will complete before commands sent to an IDE drive as: > > [ ][ ][ ][ ][ ][ ] > > Because: > > [ ] > > is smaller than: > > [ ] > > The only remaining issue is how much overlap does latency through > a vastly slower (than the CPU clock) memory and I/O bus introduces; > I am willing to give you that it may be a smaller overlap, i.e.: > > [ ] > [ ] > > But on a loaded system (i.e., more than one process -- hence the > useless nature of microbenchmarks), *any* overlap is amplified. My point is, that even if you believe in this much overlap, you still find that overlapping N=6 operations that each take this much [--------] time in isolation will look like this overlapped: [--------] [-------------] [------------------] [-----------------------] [----------------------------] [---------------------------------] and take this much time: [-------------------------------------------] which is an improvement, but nowhere near N=6 times smaller than: [--------][--------][--------][--------][--------][--------] Furthermore, a reasonable SCSI per-command-overhead is less than a millisecond while a typical disk combined-head-motion-rotational latency is on the order of 10 milliseconds. With the exception of near-but-not- actually-contiguous disk transfers (which I suspect are the exception rather than the rule), an untagged SCSI driver using the same I/O ordering rules as the disk should be able to obtain very nearly the same I/O performance as a tagged SCSI disk driver. When disk latencies are relatively high (as for randomly scattered I/O operations) the relative performances should be almost identical. The one big advantage of tagged drivers is the possibility that disk activity could overlap dma, but this of course depends on the smarts of the particular disk drive and the SCSI host adapter and it only matters if the disk latencies are so small that disk revs would be lost otherwise. (It is hard to draw a picure of this in ascii.) Even in this case, a smart driver that does 2 simultaneous SCSI commands might do as well as one that does 64. This also applies to the special case of doing contiguous disk reads from a drive that does substantial read-ahead. There is no lost-rev issue, but overlapping dma with something else is possible. In this case also, 2 simultaneous SCSI commands are probably as good as 64 performance improvements over the smart untagged driver cannot possibly exceed a factor of two. (Why are we beating this horse in freebsd-smp? I don't know. This is one of those off-topic meanderings. Perhaps we should instead inflict this exchange on freebsd-scsi. Perhaps we should just drop the subject. I suspect that we are somehow trying to compare apples and oranges.) Dan Strick dan@math.berkeley.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 9 14:23:44 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id OAA23709 for freebsd-smp-outgoing; Fri, 9 Oct 1998 14:23:44 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id OAA23683 for ; Fri, 9 Oct 1998 14:23:32 -0700 (PDT) (envelope-from tlambert@usr01.primenet.com) Received: (from daemon@localhost) by smtp04.primenet.com (8.8.8/8.8.8) id OAA07393; Fri, 9 Oct 1998 14:23:22 -0700 (MST) Received: from usr01.primenet.com(206.165.6.201) via SMTP by smtp04.primenet.com, id smtpd007361; Fri Oct 9 14:23:14 1998 Received: (from tlambert@localhost) by usr01.primenet.com (8.8.5/8.8.5) id OAA26437; Fri, 9 Oct 1998 14:23:11 -0700 (MST) From: Terry Lambert Message-Id: <199810092123.OAA26437@usr01.primenet.com> Subject: Re: hw platform Q - what's a good smp choice these days? To: dan@math.berkeley.edu (Dan Strick) Date: Fri, 9 Oct 1998 21:23:10 +0000 (GMT) Cc: tlambert@primenet.com, dan@math.berkeley.edu, freebsd-smp@FreeBSD.ORG In-Reply-To: <199810090714.AAA02765@math.berkeley.edu> from "Dan Strick" at Oct 9, 98 00:14:12 am X-Mailer: ELM [version 2.4 PL25] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > and take this much time: > > [-------------------------------------------] > > which is an improvement, but nowhere near N=6 times smaller than: > > [--------][--------][--------][--------][--------][--------] > > Furthermore, a reasonable SCSI per-command-overhead is less than a > millisecond while a typical disk combined-head-motion-rotational latency > is on the order of 10 milliseconds. With the exception of near-but-not- > actually-contiguous disk transfers (which I suspect are the exception > rather than the rule), an untagged SCSI driver using the same I/O > ordering rules as the disk should be able to obtain very nearly the > same I/O performance as a tagged SCSI disk driver. When disk latencies > are relatively high (as for randomly scattered I/O operations) the > relative performances should be almost identical. We can argue about whether the FS code should be reading mode page 2 and acting with the physical geometry in mind in order to minimize actual seeks, and that FreeBSD's imaginary 4M cylinder is broken. However. The clustering code and locality of reference will generally ensure that data locality for a given process is relatively high; this, combined with the fact that most modern SCSI drives inverse-order the blocks and start reading immediately after the seek (effectively read-caching the hicgh locality data) will definitely shorten the increment. Also, let me once again emphasize that the improvement is due to interleaved I/O, which gets rid of the agregate latency, replacing it with a single latency. > The one big advantage of tagged drivers is the possibility that disk > activity could overlap dma, but this of course depends on the smarts of > the particular disk drive and the SCSI host adapter and it only matters > if the disk latencies are so small that disk revs would be lost > otherwise. (It is hard to draw a picure of this in ascii.) Even in this > case, a smart driver that does 2 simultaneous SCSI commands might do > as well as one that does 64. If there is an intervening seek, yes. But in general, the number of sectors per cylinder has increased, not decreased, over time. We can also state that a process waiting for I/O will be involuntarily context switched, in favor of another process, and that the pool retention time that we are really interested, in terms of determining overall data transfer rates, is based on the transfer to user space, not merely the transfer from the controller into system memory. As before, this greatly amplifies the effects of serialized I/O, hence my initial steep slope for my "stair-step" diagram. > This also applies to the special case of doing contiguous disk reads > from a drive that does substantial read-ahead. There is no lost-rev > issue, but overlapping dma with something else is possible. In this > case also, 2 simultaneous SCSI commands are probably as good as 64 > performance improvements over the smart untagged driver cannot possibly > exceed a factor of two. I can't really parse this, but if (1) the commands are overlapped, and (2) operating against read-ahead cache on the disk itself, then I can't see how more commands don't equal more performance, in terms of linearly scaling. I don't think that it's likely, unless the disk itself contains as many track buffers as some high fraction of the number of tagged commands it supports (in the ideal, 1:1), to achive optimal benefit, but it's certainly unlikely to be as pessimal as taking a seek hit plus a rotational latency, which is what your "2" implies... > (Why are we beating this horse in freebsd-smp? I don't know. > This is one of those off-topic meanderings. Perhaps we should > instead inflict this exchange on freebsd-scsi. Perhaps we should > just drop the subject. I suspect that we are somehow trying to > compare apples and oranges.) Or "IDE and SCSI under load instead of under lmbench and/or some other single-process disk speed benchmark, like iozone". 8-). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Oct 9 22:03:02 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA14102 for freebsd-smp-outgoing; Fri, 9 Oct 1998 22:03:02 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from math.berkeley.edu (math.Berkeley.EDU [128.32.183.94]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA14096 for ; Fri, 9 Oct 1998 22:02:55 -0700 (PDT) (envelope-from dan@math.berkeley.edu) Received: (from dan@localhost) by math.berkeley.edu (8.8.7/8.8.7) id WAA17750; Fri, 9 Oct 1998 22:02:44 -0700 (PDT) Date: Fri, 9 Oct 1998 22:02:44 -0700 (PDT) From: dan@math.berkeley.edu (Dan Strick) Message-Id: <199810100502.WAA17750@math.berkeley.edu> To: tlambert@primenet.com Subject: Re: hw platform Q - what's a good smp choice these days? Cc: dan@math.berkeley.edu, freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > We can argue about whether the FS code should be reading mode page 2 > and acting with the physical geometry in mind in order to minimize > actual seeks, and that FreeBSD's imaginary 4M cylinder is broken. I suspect we would both agree that teaching the FS code and the driver code to optimize for the actual disk geometry would be so painful and perhaps computationally expensive as to be not worth the effort. It is probably adequate to model a disk as a sequence of blocks with on the average a much larger latency between nonconsecutive blocks than between consecutive blocks. It may also be true that on the average the latency increases with the difference in block numbers, but the actual function is so jagged that this approximation is of uncertain value. I tend to divide disk activity into several categories which must be optimized separately. The first category is randomly located I/O requests separated by large disk latencies. In this case, I/O reordering is useful and the per command SCSI latencies are so small that they fit unnoticed within the large disk latencies. There is no practical difference between reads and writes. Simultaneously executed SCSI commands are not very useful. The second category is highly localized disk I/O for mostly noncontiguous chunks. The third category is contiguous disk I/O. Both of these categories have read and write cases. Since modern drives do speculative read ahead, the read cases behave similarly, but the the write cases are different. (Modern drives may also be capable of write-behind (i.e. cached writes) but they had better not do it with my valued data.) > The clustering code and locality of reference will generally ensure > that data locality for a given process is relatively high; this, > combined with the fact that most modern SCSI drives inverse-order > the blocks and start reading immediately after the seek (effectively > read-caching the hicgh locality data) will definitely shorten the > increment. > > Also, let me once again emphasize that the improvement is due to > interleaved I/O, which gets rid of the agregate latency, replacing > it with a single latency. You have lost me here. I must not understand the "aggregate latency" to which you refer. If we execute our I/O commands serially, we can divide the time into SCSI command latency (command processing overhead), disk latency (waiting for the heads to reach the data), disk data transfer time (between the heads and the drive data buffer), and DMA (between the drive and system main memory through the SCSI and PCI busses). A smart drive might overlap some of the disk data transfer with DMA and in the case of highly localized disk reads it might effectively overlap disk latency with disk data transfer by doing speculative read ahead. I don't understand what you mean by "interleaved I/O" or how this relates to the I/O sub-activities I have listed above. > > The one big advantage of tagged drivers is the possibility that disk > > activity could overlap DMA, but this of course depends on the smarts of > > the particular disk drive and the SCSI host adapter and it only matters > > if the disk latencies are so small that disk revs would be lost > > otherwise. (It is hard to draw a picture of this in ascii.) Even in this > > case, a smart driver that does 2 simultaneous SCSI commands might do > > as well as one that does 64. > > If there is an intervening seek, yes. But in general, the number > of sectors per cylinder has increased, not decreased, over time. Actually, I was visualizing disk writes to consecutive or nearly consecutive sectors with no intervening seeks or head switches at all. I was also visualizing the disk sectors written in order of increasing sector number so that the disk could be kept continuously busy providing that DMA is always completed before the next sector to be written comes underneath the heads. In this case, it could be very useful to begin DMA for the next write command before the current write command is complete. Two simultaneous SCSI I/O commands might be sufficient. Reversing the sector order in the track changes everything. Without detailed knowledge of the actual disk geometry, the only obvious tactic is to issue large writes (by merging I/O requests). It doesn't much matter if you do this with a single SCSI command or a bunch of simultaneous SCSI commands. If you do it with a single command, you have reason to hope that even a dumb drive will do all the sectors in a single track in a single rev and you are certain to eliminate some of the per-command overhead though you will also force all of the merged I/O requests to wait until the last is done. If you do it with multiple SCSI commands, you might benefit from early completion of some of the commands. On the other hand, the drive might choose to do the I/O inefficiently. > We can also state that a process waiting for I/O will be involuntarily > context switched, in favor of another process, and that the pool > retention time that we are really interested, in terms of determining > overall data transfer rates, is based on the transfer to user space, > not merely the transfer from the controller into system memory. As > before, this greatly amplifies the effects of serialized I/O, hence > my initial steep slope for my "stair-step" diagram. I think you are saying that the process of transferring data between wherever the device controller accesses it and the running program's virtual memory is something else that can be overlapped with the other I/O activities if only we are doing enough different things at once. I would guess that this transfer process takes place at least at main memory speeds, something on the order of 10 times the raw disk data transfer rate. I suspect the memory transfer latency can almost be ignored. (I also don't understand the significance of the context switch. Perhaps I don't understand something important about the PC I/O system.) > > This also applies to the special case of doing contiguous disk reads > > from a drive that does substantial read-ahead. There is no lost-rev > > issue, but overlapping DMA with something else is possible. In this > > case also, 2 simultaneous SCSI commands are probably as good as 64 > > performance improvements over the smart untagged driver cannot possibly > > exceed a factor of two. > > I can't really parse this, but if (1) the commands are overlapped, > and (2) operating against read-ahead cache on the disk itself, > then I can't see how more commands don't equal more performance, The bottleneck will probably be the raw disk (perhaps 10 MB/sec). The SCSI bus will probably be much faster and DMA will be much faster still. Even executing only one SCSI command at a time, all this additional activity and miscellaneous SCSI command overhead, even if serialized, will mostly overlap the raw disk transfer time. Example (1 8kb transfer): SCSI bus @ 20 MB/sec: 400 us raw disk @ 10 MB/sec: 800 us PCI DMA @ 120 MB/sec: 70 us SCSI command overhead: 500 us ------------------------------- ------------------------------ total: 970 us 800 us Note: "SCSI command overhead" includes time spent in the SCSI driver. Even so, it may be overstated. It does not include time which would overlap the SCSI bus transfer or the DMA (for the same SCSI command). It is not clear how much if any of the DMA overlaps the SCSI bus transfer. This overlap is not affected by using tagged SCSI commands. > in terms of linearly scaling. I don't think that it's likely, > unless the disk itself contains as many track buffers as some > high fraction of the number of tagged commands it supports (in > the ideal, 1:1), to achive optimal benefit, but it's certainly > unlikely to be as pessimal as taking a seek hit plus a rotational > latency, which is what your "2" implies... I don't think I mentioned seeks or rotational latencies in this case. My model assumes the disk drive is sucking bits off the disk as fast as they come and that the disk drive is passing those bits down the SCSI bus to the host adapter as fast as it asks for them. The raw disk activity is basically read-ahead. It cannot overlap itself. It happens continuously, even if the SCSI read commands are executed one at a time. The only affect of queuing multiple simultaneous SCSI commands is to possibly overlap the "SCSI command overhead" of one command with the DMA and SCSI bus data transfer of other commands. If complete overlap were achieved, the larger would remain. (Hence the limiting factor of "2".) In this case, the raw disk transfer rate limitation would also remain, so for these numbers the best improvement would be a factor of 970/800 (about 1.2). Dan Strick dan@math.berkeley.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 16:11:51 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA13547 for freebsd-smp-outgoing; Sat, 10 Oct 1998 16:11:51 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from silverback.gorilla.net (silverback.gorilla.net [208.128.8.1]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id QAA13542 for ; Sat, 10 Oct 1998 16:11:48 -0700 (PDT) (envelope-from tom@gorilla.net) Received: from peeper.TOJ.org [208.143.84.7] by silverback.gorilla.net with ESMTP (SMTPD32-4.06) id A8AF31001BE; Sat, 10 Oct 1998 18:07:27 CDT Received: from slim.TOJ.org (slim.TOJ.org [192.168.0.5]) by peeper.TOJ.org (8.9.1/8.8.8) with ESMTP id SAA00355 for ; Sat, 10 Oct 1998 18:11:36 -0500 (CDT) (envelope-from tom@gorilla.net) Received: (from tom@localhost) by slim.TOJ.org (8.9.1/8.8.8) id SAA00268 for freebsd-smp@freebsd.org; Sat, 10 Oct 1998 18:11:04 -0500 (CDT) (envelope-from tom) Message-ID: <19981010181103.A261@TOJ.org> Date: Sat, 10 Oct 1998 18:11:03 -0500 From: Tom Jackson To: freebsd-smp@FreeBSD.ORG Subject: reboot no work Reply-To: toj@silverback.gorilla.net Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org With a right up to the moment beta, is anybody else seeing a fastboot or reboot locking up tight right after syncing? No keyboard led action nothing, requires the ole reset button and does not say improper unmount of /. Tom -- IMail Server for Windows NT. Evaluation version. Copyright (c) 1995-98 Ipswitch, Inc. http://www.ipswitch.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 16:25:25 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id QAA14625 for freebsd-smp-outgoing; Sat, 10 Oct 1998 16:25:25 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from odin.globalaxis.com (www2.globalaxis.com [209.141.162.5]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id QAA14613 for ; Sat, 10 Oct 1998 16:25:21 -0700 (PDT) (envelope-from x@compucore.net) Received: from logikal (ppp116.globalaxis.com [209.141.162.116]) by odin.globalaxis.com (8.8.7/8.8.7) with SMTP id TAA08311; Sat, 10 Oct 1998 19:23:50 -0400 Message-Id: <199810102323.TAA08311@odin.globalaxis.com> X-Sender: x@mail.compucore.net X-Mailer: QUALCOMM Windows Eudora Pro Version 4.0.1 Date: Sat, 10 Oct 1998 19:25:55 -0400 To: toj@silverback.gorilla.net, freebsd-smp@FreeBSD.ORG From: "Prof. Stephen W. Falken" Subject: Re: reboot no work In-Reply-To: <19981010181103.A261@TOJ.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >With a right up to the moment beta, is anybody else seeing >a fastboot or reboot locking up tight right after syncing? Do you really think this problem is related to SMP? - Stephen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 17:36:24 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id RAA21128 for freebsd-smp-outgoing; Sat, 10 Oct 1998 17:36:24 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from silverback.gorilla.net (silverback.gorilla.net [208.128.8.1]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id RAA21123 for ; Sat, 10 Oct 1998 17:36:22 -0700 (PDT) (envelope-from tom@gorilla.net) Received: from peeper.TOJ.org [208.143.84.74] by silverback.gorilla.net with ESMTP (SMTPD32-4.06) id AC625290230; Sat, 10 Oct 1998 19:31:30 CDT Received: (from tom@localhost) by peeper.TOJ.org (8.9.1/8.8.8) id TAA00386; Sat, 10 Oct 1998 19:35:39 -0500 (CDT) (envelope-from tom) Message-ID: <19981010193539.A349@TOJ.org> Date: Sat, 10 Oct 1998 19:35:39 -0500 From: Tom Jackson To: "Prof. Stephen W. Falken" , freebsd-smp@FreeBSD.ORG Subject: Re: reboot no work References: <19981010181103.A261@TOJ.org> <199810102323.TAA08311@odin.globalaxis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <199810102323.TAA08311@odin.globalaxis.com>; from Prof. Stephen W. Falken on Sat, Oct 10, 1998 at 07:25:55PM -0400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Thanks for asking, very good question. Well it doesn't show up on my Thinkpad using the same bytes from the server and kernel compiled on the server. It gets more interesting, or is that perplexing? To make sure, I tried to boot the server with a single cpu kernel made at the same time. It went into single user because it can't fsck sd1. It does reboot from that condition. When in single user mode both fdisk and disk- label report that 2 disk without msdos partitions are not configured (?). They report as expected in smp mode. At this point, I'm not really sure if I have a damaged system or there is a bug somewhere. I'll keep looking. Hopefully someone might recognise something familar. On Sat, Oct 10, 1998 at 07:25:55PM -0400, Prof. Stephen W. Falken wrote: > >With a right up to the moment beta, is anybody else seeing > >a fastboot or reboot locking up tight right after syncing? > > Do you really think this problem is related to SMP? > > - Stephen > -- Tom -- IMail Server for Windows NT. Evaluation version. Copyright (c) 1995-98 Ipswitch, Inc. http://www.ipswitch.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 18:08:07 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id SAA24867 for freebsd-smp-outgoing; Sat, 10 Oct 1998 18:08:07 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from panzer.plutotech.com (panzer.plutotech.com [206.168.67.125]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id SAA24860 for ; Sat, 10 Oct 1998 18:08:04 -0700 (PDT) (envelope-from ken@panzer.plutotech.com) Received: (from ken@localhost) by panzer.plutotech.com (8.9.1/8.8.5) id TAA09709; Sat, 10 Oct 1998 19:07:37 -0600 (MDT) From: "Kenneth D. Merry" Message-Id: <199810110107.TAA09709@panzer.plutotech.com> Subject: Re: reboot no work In-Reply-To: <19981010193539.A349@TOJ.org> from Tom Jackson at "Oct 10, 98 07:35:39 pm" To: toj@gorilla.net (Tom Jackson) Date: Sat, 10 Oct 1998 19:07:37 -0600 (MDT) Cc: x@compucore.net, freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL28s (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Tom Jackson wrote... > Thanks for asking, very good question. Well it doesn't show up on my > Thinkpad using the same bytes from the server and kernel compiled on the > server. It gets more interesting, or is that perplexing? > > To make sure, I tried to boot the server with a single cpu kernel made at > the same time. It went into single user because it can't fsck sd1. It does > reboot from that condition. When in single user mode both fdisk and disk- > label report that 2 disk without msdos partitions are not configured (?). > They report as expected in smp mode. At this point, I'm not really sure if > I have a damaged system or there is a bug somewhere. I'll keep looking. > Hopefully someone might recognise something familar. Can you pop it into the debugger when it hangs? A stack trace might help diagnose the problem. Ken -- Kenneth Merry ken@plutotech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 19:43:47 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id TAA01186 for freebsd-smp-outgoing; Sat, 10 Oct 1998 19:43:47 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from silverback.gorilla.net (silverback.gorilla.net [208.128.8.1]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id TAA01181 for ; Sat, 10 Oct 1998 19:43:45 -0700 (PDT) (envelope-from tom@gorilla.net) Received: from peeper.TOJ.org [208.143.84.80] by silverback.gorilla.net with ESMTP (SMTPD32-4.06) id AA501720286; Sat, 10 Oct 1998 21:39:12 CDT Received: (from tom@localhost) by peeper.TOJ.org (8.9.1/8.8.8) id VAA00368; Sat, 10 Oct 1998 21:42:21 -0500 (CDT) (envelope-from tom) Message-ID: <19981010214101.A327@TOJ.org> Date: Sat, 10 Oct 1998 21:41:01 -0500 From: Tom Jackson To: "Kenneth D. Merry" Cc: freebsd-smp@FreeBSD.ORG Subject: Re: reboot no work References: <19981010193539.A349@TOJ.org> <199810110107.TAA09709@panzer.plutotech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <199810110107.TAA09709@panzer.plutotech.com>; from Kenneth D. Merry on Sat, Oct 10, 1998 at 07:07:37PM -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sat, Oct 10, 1998 at 07:07:37PM -0600, Kenneth D. Merry wrote: > Tom Jackson wrote... > > Thanks for asking, very good question. Well it doesn't show up on my > > Thinkpad using the same bytes from the server and kernel compiled on the > > server. It gets more interesting, or is that perplexing? > > > > To make sure, I tried to boot the server with a single cpu kernel made at > > the same time. It went into single user because it can't fsck sd1. It does > > reboot from that condition. When in single user mode both fdisk and disk- > > label report that 2 disk without msdos partitions are not configured (?). > > They report as expected in smp mode. At this point, I'm not really sure if > > I have a damaged system or there is a bug somewhere. I'll keep looking. > > Hopefully someone might recognise something familar. > > Can you pop it into the debugger when it hangs? > > A stack trace might help diagnose the problem. > > Ken > -- > Kenneth Merry > ken@plutotech.com > Sorry, no. THe keyboard is froze and only the reset button will bring me back. Have tried turning off all local script startups but still have problem. If I don't wait a respectable time after issuing the reboot I get a improper umount into fsck land. Have also noticed a somewhat long time delay after local startups when booting. This reboot problem has only been around since about the 10/8 or 10/9. -- Tom -- IMail Server for Windows NT. Evaluation version. Copyright (c) 1995-98 Ipswitch, Inc. http://www.ipswitch.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 19:49:31 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id TAA01592 for freebsd-smp-outgoing; Sat, 10 Oct 1998 19:49:31 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from gauss.ee.calpoly.edu (gauss.ee.calpoly.edu [129.65.26.111]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id TAA01582 for ; Sat, 10 Oct 1998 19:49:27 -0700 (PDT) (envelope-from kwayman@gauss.ee.calpoly.edu) Received: (from kwayman@localhost) by gauss.ee.calpoly.edu (8.8.8/8.6.12) id TAA28421 for freebsd-smp@FreeBSD.ORG; Sat, 10 Oct 1998 19:40:35 -0700 (PDT) From: Kyle Wayman Message-Id: <199810110240.TAA28421@gauss.ee.calpoly.edu> Subject: Re: reboot no work In-Reply-To: <19981010193539.A349@TOJ.org> from Tom Jackson at "Oct 10, 98 07:35:39 pm" To: freebsd-smp@FreeBSD.ORG Date: Sat, 10 Oct 1998 19:40:35 -0700 (PDT) X-Mailer: ELM [version 2.4ME+ PL40 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >With a right up to the moment beta, is anybody else seeing > >a fastboot or reboot locking up tight right after syncing? > > Do you really think this problem is related to SMP? This is a good question because I have had this problem, but only once. The machine that it happened to was a single processor machine though. It was a SNAP Aug 4. ----------------------------------------------------------------- = Kyle Wayman kwayman@gauss.calpoly.edu = = System Administrator office: 805-756-1390 = = EE Department, Cal Poly, SLO page: 805-542-5777 = ----------------------------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 20:08:13 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id UAA04231 for freebsd-smp-outgoing; Sat, 10 Oct 1998 20:08:13 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from panzer.plutotech.com (panzer.plutotech.com [206.168.67.125]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id UAA04226 for ; Sat, 10 Oct 1998 20:08:11 -0700 (PDT) (envelope-from ken@panzer.plutotech.com) Received: (from ken@localhost) by panzer.plutotech.com (8.9.1/8.8.5) id VAA10428; Sat, 10 Oct 1998 21:07:56 -0600 (MDT) From: "Kenneth D. Merry" Message-Id: <199810110307.VAA10428@panzer.plutotech.com> Subject: Re: reboot no work In-Reply-To: <19981010214101.A327@TOJ.org> from Tom Jackson at "Oct 10, 98 09:41:01 pm" To: toj@gorilla.net (Tom Jackson) Date: Sat, 10 Oct 1998 21:07:56 -0600 (MDT) Cc: freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL28s (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Tom Jackson wrote... > On Sat, Oct 10, 1998 at 07:07:37PM -0600, Kenneth D. Merry wrote: > > Tom Jackson wrote... > > > Thanks for asking, very good question. Well it doesn't show up on my > > > Thinkpad using the same bytes from the server and kernel compiled on the > > > server. It gets more interesting, or is that perplexing? > > > > > > To make sure, I tried to boot the server with a single cpu kernel made at > > > the same time. It went into single user because it can't fsck sd1. It does > > > reboot from that condition. When in single user mode both fdisk and disk- > > > label report that 2 disk without msdos partitions are not configured (?). > > > They report as expected in smp mode. At this point, I'm not really sure if > > > I have a damaged system or there is a bug somewhere. I'll keep looking. > > > Hopefully someone might recognise something familar. > > > > Can you pop it into the debugger when it hangs? > > > > A stack trace might help diagnose the problem. > > > > Ken > > -- > > Kenneth Merry > > ken@plutotech.com > > > Sorry, no. THe keyboard is froze and only the reset button will bring > me back. Have tried turning off all local script startups but still > have problem. If I don't wait a respectable time after issuing the reboot > I get a improper umount into fsck land. Have also noticed a somewhat long > time delay after local startups when booting. This reboot problem has only > been around since about the 10/8 or 10/9. Can you try backing sys/cam/scsi/scsi_da.c to revision 1.7? It could be that the cache sync command is causing one of your disks to hang. I'm not sure that's the problem, though, since you should be able to get into the debugger when it hangs. Also, what kind of SCSI controller do you have? There was a bug in the aha, bt and ahb drivers that caused the polling-for-completion stuff to not work. Ken -- Kenneth Merry ken@plutotech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 20:42:47 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id UAA08028 for freebsd-smp-outgoing; Sat, 10 Oct 1998 20:42:47 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from kapmail.com (srv.kapmail.com [206.31.219.210]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id UAA08015 for ; Sat, 10 Oct 1998 20:42:44 -0700 (PDT) (envelope-from satya@dspsoft.com) Received: (qmail 2276 invoked from network); 11 Oct 1998 03:43:50 -0000 Received: from simba.dspsoft.com (206.31.219.210) by simba.dspsoft.com with SMTP; 11 Oct 1998 03:43:50 -0000 Date: Sat, 10 Oct 1998 23:43:50 -0400 (EDT) From: Satya Devireddy X-Sender: satya@srv.kapmail.com To: freebsd-smp@FreeBSD.ORG Subject: Re: reboot no work In-Reply-To: <19981010214101.A327@TOJ.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Can you pop it into the debugger when it hangs? > > > > A stack trace might help diagnose the problem. > > > Sorry, no. THe keyboard is froze and only the reset button will bring > me back. Have tried turning off all local script startups but still > have problem. If I don't wait a respectable time after issuing the reboot > I get a improper umount into fsck land. Have also noticed a somewhat long > time delay after local startups when booting. This reboot problem has only > been around since about the 10/8 or 10/9. I too am having a similar problem... I cannot seem to build a working kernel for about a day. My previous working kernel was about 9 days old. The new kernels seem to hang immeditely after the boot prompt after they display that code addresses (i think) No output of the devices recognized show up on the screen. But I can see my floppy drive, cd, hard disks light up. If I reboot at this time and boot using the saved kernle (9 days old), it fscks / and all as they are not properly dismounted. Somehow it seems to be doing all the device detection and probably mounting the disks but the system does not respond. PII 233 ASUS P2BS (7890) CURRENT (re cvsuped last night) and a clean /usr/src SOFTUPDATES (I do not think this is causing it) I did not touch any hardware recently. I think it is a problem in the boot code. I do have FFS_ROOT in the kernel config. I can provide more info if needed... -Satya To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 20:59:23 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id UAA09624 for freebsd-smp-outgoing; Sat, 10 Oct 1998 20:59:23 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from silverback.gorilla.net (silverback.gorilla.net [208.128.8.1]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id UAA09249 for ; Sat, 10 Oct 1998 20:55:38 -0700 (PDT) (envelope-from tom@gorilla.net) Received: from peeper.TOJ.org [208.143.84.86] by silverback.gorilla.net with ESMTP (SMTPD32-4.06) id AB20139029E; Sat, 10 Oct 1998 22:50:56 CDT Received: (from tom@localhost) by peeper.TOJ.org (8.9.1/8.8.8) id WAA00341; Sat, 10 Oct 1998 22:55:05 -0500 (CDT) (envelope-from tom) Message-ID: <19981010225435.A327@TOJ.org> Date: Sat, 10 Oct 1998 22:54:35 -0500 From: Tom Jackson To: "Kenneth D. Merry" Cc: freebsd-smp@FreeBSD.ORG Subject: Re: reboot no work References: <19981010214101.A327@TOJ.org> <199810110307.VAA10428@panzer.plutotech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.93.2i In-Reply-To: <199810110307.VAA10428@panzer.plutotech.com>; from Kenneth D. Merry on Sat, Oct 10, 1998 at 09:07:56PM -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sat, Oct 10, 1998 at 09:07:56PM -0600, Kenneth D. Merry wrote: > Tom Jackson wrote... > > On Sat, Oct 10, 1998 at 07:07:37PM -0600, Kenneth D. Merry wrote: > > > Tom Jackson wrote... > > > > Thanks for asking, very good question. Well it doesn't show up on my > > > > Thinkpad using the same bytes from the server and kernel compiled on the > > > > server. It gets more interesting, or is that perplexing? > > > > > > > > To make sure, I tried to boot the server with a single cpu kernel made at > > > > the same time. It went into single user because it can't fsck sd1. It does > > > > reboot from that condition. When in single user mode both fdisk and disk- > > > > label report that 2 disk without msdos partitions are not configured (?). > > > > They report as expected in smp mode. At this point, I'm not really sure if > > > > I have a damaged system or there is a bug somewhere. I'll keep looking. > > > > Hopefully someone might recognise something familar. > > > > > > Can you pop it into the debugger when it hangs? > > > > > > A stack trace might help diagnose the problem. > > > > > > Ken > > > -- > > > Kenneth Merry > > > ken@plutotech.com > > > > > Sorry, no. THe keyboard is froze and only the reset button will bring > > me back. Have tried turning off all local script startups but still > > have problem. If I don't wait a respectable time after issuing the reboot > > I get a improper umount into fsck land. Have also noticed a somewhat long > > time delay after local startups when booting. This reboot problem has only > > been around since about the 10/8 or 10/9. > > Can you try backing sys/cam/scsi/scsi_da.c to revision 1.7? It could be > that the cache sync command is causing one of your disks to hang. > > I'm not sure that's the problem, though, since you should be able to get > into the debugger when it hangs. > > Also, what kind of SCSI controller do you have? There was a bug in the > aha, bt and ahb drivers that caused the polling-for-completion stuff to not > work. > > Ken > -- > Kenneth Merry > ken@plutotech.com > YES, that seems to have fixed it, at least for me. The controller is an onboard aic7880 wide ultra. There's also a 2940 narrow in the box. The two main drives are Fujitsi M2954[SQ]. Justin looked at the Q this summer. What's next? -- Tom -- IMail Server for Windows NT. Evaluation version. Copyright (c) 1995-98 Ipswitch, Inc. http://www.ipswitch.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 21:15:34 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA11638 for freebsd-smp-outgoing; Sat, 10 Oct 1998 21:15:34 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from panzer.plutotech.com (panzer.plutotech.com [206.168.67.125]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id VAA11633 for ; Sat, 10 Oct 1998 21:15:32 -0700 (PDT) (envelope-from ken@panzer.plutotech.com) Received: (from ken@localhost) by panzer.plutotech.com (8.9.1/8.8.5) id WAA11061; Sat, 10 Oct 1998 22:15:19 -0600 (MDT) From: "Kenneth D. Merry" Message-Id: <199810110415.WAA11061@panzer.plutotech.com> Subject: Re: reboot no work In-Reply-To: <19981010225435.A327@TOJ.org> from Tom Jackson at "Oct 10, 98 10:54:35 pm" To: toj@gorilla.net (Tom Jackson) Date: Sat, 10 Oct 1998 22:15:19 -0600 (MDT) Cc: freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL28s (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Tom Jackson wrote... > On Sat, Oct 10, 1998 at 09:07:56PM -0600, Kenneth D. Merry wrote: > > Can you try backing sys/cam/scsi/scsi_da.c to revision 1.7? It could be > > that the cache sync command is causing one of your disks to hang. > > > > I'm not sure that's the problem, though, since you should be able to get > > into the debugger when it hangs. > > > > Also, what kind of SCSI controller do you have? There was a bug in the > > aha, bt and ahb drivers that caused the polling-for-completion stuff to not > > work. > > > > Ken > > -- > > Kenneth Merry > > ken@plutotech.com > > > > YES, that seems to have fixed it, at least for me. The controller is an > onboard aic7880 wide ultra. There's also a 2940 narrow in the box. The > two main drives are Fujitsi M2954[SQ]. Justin looked at the Q this summer. > > What's next? Send me the dmesg output for your drives, and I'll add them into the quirk table in the DA driver so we don't try to sync the cache. Ken -- Kenneth Merry ken@plutotech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 21:20:35 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA12166 for freebsd-smp-outgoing; Sat, 10 Oct 1998 21:20:35 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from kapmail.com (srv.kapmail.com [206.31.219.210]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id VAA12152 for ; Sat, 10 Oct 1998 21:20:23 -0700 (PDT) (envelope-from satya@dspsoft.com) Received: (qmail 2596 invoked from network); 11 Oct 1998 04:21:34 -0000 Received: from simba.dspsoft.com (206.31.219.210) by simba.dspsoft.com with SMTP; 11 Oct 1998 04:21:34 -0000 Date: Sun, 11 Oct 1998 00:21:34 -0400 (EDT) From: Satya Devireddy X-Sender: satya@srv.kapmail.com To: freebsd-smp@FreeBSD.ORG Subject: Re: reboot no work In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I Fixed my problem too. Sorry for the noise.... Some gremlins ate "device sc0 ..." from my kernel config file... My mistake when trying to add FFS_ROOT ... -Satya > No output of the devices recognized show up on the screen. But I can > see my floppy drive, cd, hard disks light up. > If I reboot at this time and boot using the saved kernle (9 days old), > it fscks / and all as they are not properly dismounted. > > Somehow it seems to be doing all the device detection and probably > mounting the disks but the system does not respond. > > PII 233 ASUS P2BS (7890) > CURRENT (re cvsuped last night) and a clean /usr/src > SOFTUPDATES (I do not think this is causing it) > > I did not touch any hardware recently. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 21:34:35 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id VAA13662 for freebsd-smp-outgoing; Sat, 10 Oct 1998 21:34:35 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from panzer.plutotech.com (panzer.plutotech.com [206.168.67.125]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id VAA13657 for ; Sat, 10 Oct 1998 21:34:34 -0700 (PDT) (envelope-from ken@panzer.plutotech.com) Received: (from ken@localhost) by panzer.plutotech.com (8.9.1/8.8.5) id WAA11212; Sat, 10 Oct 1998 22:34:02 -0600 (MDT) From: "Kenneth D. Merry" Message-Id: <199810110434.WAA11212@panzer.plutotech.com> Subject: Re: reboot no work In-Reply-To: from Satya Devireddy at "Oct 10, 98 11:43:50 pm" To: satya@dspsoft.com (Satya Devireddy) Date: Sat, 10 Oct 1998 22:34:02 -0600 (MDT) Cc: freebsd-smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL28s (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Satya Devireddy wrote... > > > Can you pop it into the debugger when it hangs? > > > > > > A stack trace might help diagnose the problem. > > > > > Sorry, no. THe keyboard is froze and only the reset button will bring > > me back. Have tried turning off all local script startups but still > > have problem. If I don't wait a respectable time after issuing the reboot > > I get a improper umount into fsck land. Have also noticed a somewhat long > > time delay after local startups when booting. This reboot problem has only > > been around since about the 10/8 or 10/9. > > I too am having a similar problem... > I cannot seem to build a working kernel for about a day. My previous > working kernel was about 9 days old. > > The new kernels seem to hang immeditely after the boot prompt after they > display that code addresses (i think) > > No output of the devices recognized show up on the screen. But I can > see my floppy drive, cd, hard disks light up. > If I reboot at this time and boot using the saved kernle (9 days old), > it fscks / and all as they are not properly dismounted. > > Somehow it seems to be doing all the device detection and probably > mounting the disks but the system does not respond. Although it sounds similar, this is a different problem. Tom's problem is that one of his disks doesn't like the cache sync command. The cache sync doesn't happen until you close the disk device, or until a reboot happens. Your problem happens on boot, so it must be something else. Ken -- Kenneth Merry ken@plutotech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 22:43:00 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA19698 for freebsd-smp-outgoing; Sat, 10 Oct 1998 22:43:00 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA19690 for ; Sat, 10 Oct 1998 22:42:56 -0700 (PDT) (envelope-from gibbs@narnia.plutotech.com) Received: (from gibbs@localhost) by narnia.plutotech.com (8.9.1/8.7.3) id XAA01917; Sat, 10 Oct 1998 23:36:00 -0600 (MDT) Date: Sat, 10 Oct 1998 23:36:00 -0600 (MDT) From: "Justin T. Gibbs" Message-Id: <199810110536.XAA01917@narnia.plutotech.com> To: "Kenneth D. Merry" cc: smp@FreeBSD.ORG Subject: Re: reboot no work Newsgroups: pluto.freebsd.smp In-Reply-To: <199810110434.WAA11212@panzer.plutotech.com> User-Agent: tin/pre-1.4-971204 (UNIX) (FreeBSD/3.0-BETA (i386)) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Although it sounds similar, this is a different problem. Tom's problem is > that one of his disks doesn't like the cache sync command. I don't see how that can be. We have performed a sync cache on final device close since CAM came into -current. If you shutdown normally, the shutdown hook you added should have no effect (all devices should have seen final close). -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 22:48:43 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA20358 for freebsd-smp-outgoing; Sat, 10 Oct 1998 22:48:43 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from panzer.plutotech.com (panzer.plutotech.com [206.168.67.125]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA20353 for ; Sat, 10 Oct 1998 22:48:42 -0700 (PDT) (envelope-from ken@panzer.plutotech.com) Received: (from ken@localhost) by panzer.plutotech.com (8.9.1/8.8.5) id XAA12036; Sat, 10 Oct 1998 23:48:31 -0600 (MDT) From: "Kenneth D. Merry" Message-Id: <199810110548.XAA12036@panzer.plutotech.com> Subject: Re: reboot no work In-Reply-To: <199810110536.XAA01917@narnia.plutotech.com> from "Justin T. Gibbs" at "Oct 10, 98 11:36:00 pm" To: gibbs@pluto.plutotech.com Date: Sat, 10 Oct 1998 23:48:31 -0600 (MDT) Cc: smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL28s (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Justin T. Gibbs wrote... > > Although it sounds similar, this is a different problem. Tom's problem is > > that one of his disks doesn't like the cache sync command. > > I don't see how that can be. We have performed a sync cache on final > device close since CAM came into -current. If you shutdown normally, > the shutdown hook you added should have no effect (all devices should > have seen final close). I don't think that's necessarily the case. (Although it would make sense that all devices would see final close, wouldn't it? :) When I was testing out the shutdown hook changes, I noticed that every time I rebooted, the open flag was cleared for the secondary drive, but not for the boot drive. So, every time I tried it, the cache sync for the boot drive was performed by the shutdown hook, not by daclose(). My guess is that the drive that Tom is having trouble with isn't getting final close, and that's why he hasn't seen this before. I'm not sure what the conditions are that would cause a drive not to see final close when we reboot. It sounds like it might be a bug somewhere. Ken -- Kenneth Merry ken@plutotech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 22:57:33 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id WAA21019 for freebsd-smp-outgoing; Sat, 10 Oct 1998 22:57:33 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from pluto.plutotech.com (mail.plutotech.com [206.168.67.137]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id WAA21014 for ; Sat, 10 Oct 1998 22:57:32 -0700 (PDT) (envelope-from gibbs@plutotech.com) Received: from narnia.plutotech.com (narnia.plutotech.com [206.168.67.130]) by pluto.plutotech.com (8.8.7/8.8.5) with ESMTP id XAA21231; Sat, 10 Oct 1998 23:57:21 -0600 (MDT) Message-Id: <199810110557.XAA21231@pluto.plutotech.com> X-Mailer: exmh version 2.0.2 2/24/98 To: "Kenneth D. Merry" cc: gibbs@plutotech.com, smp@FreeBSD.ORG Subject: Re: reboot no work In-reply-to: Your message of "Sat, 10 Oct 1998 23:48:31 MDT." <199810110548.XAA12036@panzer.plutotech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 10 Oct 1998 23:50:34 -0600 From: "Justin T. Gibbs" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >I don't think that's necessarily the case. (Although it would make sense >that all devices would see final close, wouldn't it? :) When I was testing >out the shutdown hook changes, I noticed that every time I rebooted, the >open flag was cleared for the secondary drive, but not for the boot drive. There must be a slice always left open. But if this is the case, the version of scsi_da that did not check in close to see if other slices were open would still see at least a close or two during boot processing. Disk devices seem to get opened and closed several times early in system startup. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Oct 10 23:01:38 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id XAA21540 for freebsd-smp-outgoing; Sat, 10 Oct 1998 23:01:38 -0700 (PDT) (envelope-from owner-freebsd-smp@FreeBSD.ORG) Received: from panzer.plutotech.com (panzer.plutotech.com [206.168.67.125]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id XAA21531 for ; Sat, 10 Oct 1998 23:01:33 -0700 (PDT) (envelope-from ken@panzer.plutotech.com) Received: (from ken@localhost) by panzer.plutotech.com (8.9.1/8.8.5) id AAA12145; Sun, 11 Oct 1998 00:01:22 -0600 (MDT) From: "Kenneth D. Merry" Message-Id: <199810110601.AAA12145@panzer.plutotech.com> Subject: Re: reboot no work In-Reply-To: <199810110557.XAA21231@pluto.plutotech.com> from "Justin T. Gibbs" at "Oct 10, 98 11:50:34 pm" To: gibbs@pluto.plutotech.com Date: Sun, 11 Oct 1998 00:01:22 -0600 (MDT) Cc: smp@FreeBSD.ORG X-Mailer: ELM [version 2.4ME+ PL28s (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Justin T. Gibbs wrote... > >I don't think that's necessarily the case. (Although it would make sense > >that all devices would see final close, wouldn't it? :) When I was testing > >out the shutdown hook changes, I noticed that every time I rebooted, the > >open flag was cleared for the secondary drive, but not for the boot drive. > > There must be a slice always left open. But if this is the case, the > version of scsi_da that did not check in close to see if other slices > were open would still see at least a close or two during boot processing. > Disk devices seem to get opened and closed several times early in system > startup. Well, he said that the problem went away when he went from version 1.8 of scsi_da.c to version 1.7. Version 1.8 was the one where I added the sync cache stuff. Any theories as to why it makes a difference? Ken -- Kenneth Merry ken@plutotech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message