From owner-freebsd-performance@FreeBSD.ORG Fri Sep 10 13:21:49 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F133916A4CE for ; Fri, 10 Sep 2004 13:21:49 +0000 (GMT) Received: from freebee.digiware.nl (dsl390.iae.nl [212.61.63.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 30E4943D41 for ; Fri, 10 Sep 2004 13:21:49 +0000 (GMT) (envelope-from wjw@withagen.nl) Received: from [212.61.27.71] (dual [212.61.27.71]) by freebee.digiware.nl (8.12.10/8.12.10) with ESMTP id i8ADLkEg072092; Fri, 10 Sep 2004 15:21:47 +0200 (CEST) (envelope-from wjw@withagen.nl) Message-ID: <4141AA6A.2070802@withagen.nl> Date: Fri, 10 Sep 2004 15:21:46 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Geert Hendrickx References: <20040831133551.GA86660@lori.mine.nu> <4134B312.8030309@pacific.net.sg> <1093958674.680.2.camel@book> <20040831183908.GA87694@lori.mine.nu> In-Reply-To: <20040831183908.GA87694@lori.mine.nu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org cc: oceanare@pacific.net.sg Subject: Re: spreading partitions over multiple drives X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Sep 2004 13:21:50 -0000 Geert Hendrickx wrote: >>Not for this usage. >> >> >>>Fragmentation may be LESS of a problem with UFS, but a moving target >>>like one big /usr (incl src, obj, ports) will get fragmented as well. >>> >>> >>This is how you see it. I have not heard that there is any tool to help >>here. >> >>I would not call this fragmentation. It is more like spreading the files >>from one directory all over the disk. >> >> > >Ok but the effect is the same: constant movement of the head. > > I would expect a bigger system to cache just about all file access during 'make buildworld'. Even when building things with -j 64 I can not get my dual-opteron 1Gb system get without free pages. And as such most files will only be read once, and object output will be "slowly" synced on the disks. Disk I/O rearly becomes the bottleneck, most of the time I'm missing raw CPU cycles. And I have everything on 1 large 200Gb disk. >>>Splitting up partitions would reduce this fragmentation (as you are >>>essentially defining some "super large blocks"), and may increase >>>filesystem stability in case of crashes etc. >>> >>> >>> >>It might not affect stability but it increases the chances to fix a >>problem in case of a crash. >> >> > >Yes I meant stability of the filesystem not of the running OS. > > >Ok but the original question was about spreading partitions amongst >multiple disks, not pro/con splitting partitions on one disk. :-) > > My major problem with a lot of partitions has always been that one way or antoher I outgrow a partition and then all of a sudden the logic needs to be skewed for space reasons. Finaly disks start to grow to the size where this becomes a moot point. 10Gb for /usr will last me until we get to the next step of disk-sizes. Having things on different spindles will of course be a major plus. --WjW From owner-freebsd-performance@FreeBSD.ORG Fri Sep 10 16:01:08 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B44D16A4CE for ; Fri, 10 Sep 2004 16:01:08 +0000 (GMT) Received: from outmx002.isp.belgacom.be (outmx002.isp.belgacom.be [195.238.3.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id CBCDB43D58 for ; Fri, 10 Sep 2004 16:01:07 +0000 (GMT) (envelope-from geert@lori.mine.nu) Received: from outmx002.isp.belgacom.be (localhost [127.0.0.1]) with ESMTP id i8AG11Ji005600 for ; Fri, 10 Sep 2004 18:01:01 +0200 (envelope-from ) Received: from lori.mine.nu (119-137.244.81.adsl.skynet.be [81.244.137.119]) with ESMTP id i8AG0pu5005492; Fri, 10 Sep 2004 18:00:51 +0200 (envelope-from ) Received: by lori.mine.nu (Postfix, from userid 1000) id 1ED3978F; Fri, 10 Sep 2004 18:00:51 +0200 (CEST) Date: Fri, 10 Sep 2004 18:00:51 +0200 From: Geert Hendrickx To: Willem Jan Withagen Message-ID: <20040910160051.GA24152@lori.mine.nu> References: <20040831133551.GA86660@lori.mine.nu> <4134B312.8030309@pacific.net.sg> <1093958674.680.2.camel@book> <20040831183908.GA87694@lori.mine.nu> <4141AA6A.2070802@withagen.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4141AA6A.2070802@withagen.nl> User-Agent: Mutt/1.4.2i X-GPG-Key: http://lori.mine.nu/gnupgkey.asc X-GPG-Key-ID: 1024D/766C1E92 X-Accept-Language: nl,en cc: freebsd-performance@freebsd.org cc: oceanare@pacific.net.sg Subject: Re: spreading partitions over multiple drives X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Sep 2004 16:01:08 -0000 On Fri, Sep 10, 2004 at 03:21:46PM +0200, Willem Jan Withagen wrote: > I would expect a bigger system to cache just about all file access > during 'make buildworld'. > Even when building things with -j 64 I can not get my dual-opteron 1Gb > system get without free pages. > And as such most files will only be read once, and object output will be > "slowly" synced on the disks. > Disk I/O rearly becomes the bottleneck, most of the time I'm missing raw > CPU cycles. > And I have everything on 1 large 200Gb disk. Ok so adding more RAM may be more useful than an extra harddisk? Maybe I could even put /tmp or /usr/obj on a RAM-disk? A fully built /usr/obj is about 350Mb. GH From owner-freebsd-performance@FreeBSD.ORG Fri Sep 10 16:45:01 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4E83E16A4CE for ; Fri, 10 Sep 2004 16:45:01 +0000 (GMT) Received: from pimout1-ext.prodigy.net (pimout1-ext.prodigy.net [207.115.63.77]) by mx1.FreeBSD.org (Postfix) with ESMTP id D486143D45 for ; Fri, 10 Sep 2004 16:45:00 +0000 (GMT) (envelope-from bob@immure.com) Received: from maul.immure.com (adsl-66-136-206-1.dsl.austtx.swbell.net [66.136.206.1])i8AGisH9248160; Fri, 10 Sep 2004 12:44:55 -0400 Received: from luke.immure.com (luke.immure.com [10.1.132.3]) by maul.immure.com (8.12.11/8.12.11) with ESMTP id i8AGiqNo010009; Fri, 10 Sep 2004 11:44:52 -0500 (CDT) (envelope-from bob@immure.com) Received: from luke.immure.com (localhost [127.0.0.1]) by luke.immure.com (8.12.11/8.12.11) with ESMTP id i8AGiqx2026519; Fri, 10 Sep 2004 11:44:52 -0500 (CDT) (envelope-from bob@luke.immure.com) Received: (from bob@localhost) by luke.immure.com (8.12.11/8.12.11/Submit) id i8AGiqmZ026518; Fri, 10 Sep 2004 11:44:52 -0500 (CDT) (envelope-from bob) Date: Fri, 10 Sep 2004 11:44:52 -0500 From: Bob Willcox To: Geert Hendrickx Message-ID: <20040910164452.GE24453@luke.immure.com> References: <20040831133551.GA86660@lori.mine.nu> <4134B312.8030309@pacific.net.sg> <1093958674.680.2.camel@book> <20040831183908.GA87694@lori.mine.nu> <4141AA6A.2070802@withagen.nl> <20040910160051.GA24152@lori.mine.nu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040910160051.GA24152@lori.mine.nu> User-Agent: Mutt/1.5.6i X-immure-MailScanner-Information: Please contact the ISP for more information X-immure-MailScanner: Found to be clean X-MailScanner-From: bob@immure.com cc: freebsd-performance@freebsd.org cc: oceanare@pacific.net.sg cc: Willem Jan Withagen Subject: Re: spreading partitions over multiple drives X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Bob Willcox List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Sep 2004 16:45:01 -0000 On Fri, Sep 10, 2004 at 06:00:51PM +0200, Geert Hendrickx wrote: > On Fri, Sep 10, 2004 at 03:21:46PM +0200, Willem Jan Withagen wrote: > > I would expect a bigger system to cache just about all file access > > during 'make buildworld'. > > Even when building things with -j 64 I can not get my dual-opteron 1Gb > > system get without free pages. > > And as such most files will only be read once, and object output will be > > "slowly" synced on the disks. > > Disk I/O rearly becomes the bottleneck, most of the time I'm missing raw > > CPU cycles. > > And I have everything on 1 large 200Gb disk. > > Ok so adding more RAM may be more useful than an extra harddisk? Maybe > I could even put /tmp or /usr/obj on a RAM-disk? A fully built /usr/obj > is about 350Mb. Hmm, how to you follow the prescribed process of rebooting between the buildworld/installkernel and installworld if you put /usr/obj on a RAM disk? Following the reboot you'll have no objects to install. Bob > > GH > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" -- Bob Willcox Acquaintance, n.: bob@immure.com A person whom we know well enough to borrow from, Austin, TX but not well enough to lend to. -- Ambrose Bierce, "The Devil's Dictionary" From owner-freebsd-performance@FreeBSD.ORG Fri Sep 10 21:08:47 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 864BE16A4CE for ; Fri, 10 Sep 2004 21:08:47 +0000 (GMT) Received: from outmx001.isp.belgacom.be (outmx001.isp.belgacom.be [195.238.3.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id F179543D58 for ; Fri, 10 Sep 2004 21:08:46 +0000 (GMT) (envelope-from geert@lori.mine.nu) Received: from outmx001.isp.belgacom.be (localhost [127.0.0.1]) with ESMTP id i8AL8cLr029381 for ; Fri, 10 Sep 2004 23:08:38 +0200 (envelope-from ) Received: from lori.mine.nu (119-137.244.81.adsl.skynet.be [81.244.137.119]) with ESMTP id i8AL8bX7029372; Fri, 10 Sep 2004 23:08:37 +0200 (envelope-from ) Received: by lori.mine.nu (Postfix, from userid 1000) id F2AC54BC; Fri, 10 Sep 2004 23:08:36 +0200 (CEST) Date: Fri, 10 Sep 2004 23:08:36 +0200 From: Geert Hendrickx To: Bob Willcox Message-ID: <20040910210836.GA28469@lori.mine.nu> References: <20040831133551.GA86660@lori.mine.nu> <4134B312.8030309@pacific.net.sg> <1093958674.680.2.camel@book> <20040831183908.GA87694@lori.mine.nu> <4141AA6A.2070802@withagen.nl> <20040910160051.GA24152@lori.mine.nu> <20040910164452.GE24453@luke.immure.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040910164452.GE24453@luke.immure.com> User-Agent: Mutt/1.4.2i X-GPG-Key: http://lori.mine.nu/gnupgkey.asc X-GPG-Key-ID: 1024D/766C1E92 X-Accept-Language: nl,en cc: freebsd-performance@freebsd.org cc: oceanare@pacific.net.sg cc: Willem Jan Withagen Subject: Re: spreading partitions over multiple drives X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Sep 2004 21:08:47 -0000 On Fri, Sep 10, 2004 at 11:44:52AM -0500, Bob Willcox wrote: > On Fri, Sep 10, 2004 at 06:00:51PM +0200, Geert Hendrickx wrote: > > On Fri, Sep 10, 2004 at 03:21:46PM +0200, Willem Jan Withagen wrote: > > > I would expect a bigger system to cache just about all file access > > > during 'make buildworld'. > > > Even when building things with -j 64 I can not get my dual-opteron 1Gb > > > system get without free pages. > > > And as such most files will only be read once, and object output will be > > > "slowly" synced on the disks. > > > Disk I/O rearly becomes the bottleneck, most of the time I'm missing raw > > > CPU cycles. > > > And I have everything on 1 large 200Gb disk. > > > > Ok so adding more RAM may be more useful than an extra harddisk? Maybe > > I could even put /tmp or /usr/obj on a RAM-disk? A fully built /usr/obj > > is about 350Mb. > > Hmm, how to you follow the prescribed process of rebooting between the > buildworld/installkernel and installworld if you put /usr/obj on a RAM > disk? Following the reboot you'll have no objects to install. > > Bob Oops! :-) GH From owner-freebsd-performance@FreeBSD.ORG Fri Sep 10 22:02:14 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC8B016A4CE for ; Fri, 10 Sep 2004 22:02:14 +0000 (GMT) Received: from freebee.digiware.nl (dsl390.iae.nl [212.61.63.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8339843D4C for ; Fri, 10 Sep 2004 22:02:13 +0000 (GMT) (envelope-from wjw@withagen.nl) Received: from [212.61.27.71] (dual [212.61.27.71]) by freebee.digiware.nl (8.12.10/8.12.10) with ESMTP id i8AM2AEg099848; Sat, 11 Sep 2004 00:02:11 +0200 (CEST) (envelope-from wjw@withagen.nl) Message-ID: <41422463.9090303@withagen.nl> Date: Sat, 11 Sep 2004 00:02:11 +0200 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Geert Hendrickx References: <20040831133551.GA86660@lori.mine.nu> <4134B312.8030309@pacific.net.sg> <1093958674.680.2.camel@book> <20040831183908.GA87694@lori.mine.nu> <4141AA6A.2070802@withagen.nl> <20040910160051.GA24152@lori.mine.nu> In-Reply-To: <20040910160051.GA24152@lori.mine.nu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org cc: oceanare@pacific.net.sg Subject: Re: spreading partitions over multiple drives X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Sep 2004 22:02:15 -0000 Geert Hendrickx wrote: >On Fri, Sep 10, 2004 at 03:21:46PM +0200, Willem Jan Withagen wrote: > > >>I would expect a bigger system to cache just about all file access >>during 'make buildworld'. >>Even when building things with -j 64 I can not get my dual-opteron 1Gb >>system get without free pages. >>And as such most files will only be read once, and object output will be >>"slowly" synced on the disks. >>Disk I/O rearly becomes the bottleneck, most of the time I'm missing raw >>CPU cycles. >>And I have everything on 1 large 200Gb disk. >> >> > >Ok so adding more RAM may be more useful than an extra harddisk? Maybe >I could even put /tmp or /usr/obj on a RAM-disk? A fully built /usr/obj >is about 350Mb. > > It would be wise to consider these things rather different. It used to be black magic in the time of Sun Files and/or work servers and I guess it has not changed all that much. Since there is now a unified buffer the disctinction between the 2 has sort of faded, but in general you need as much memory as you need for running concurrent application where you sum the size of the workset of each of the applications. Hence you prevent swapping and/or paging as much as possible. New in the equation is that that same memory know could also hold the file cache. (This used to be a hard configurable limit, set before/whilest booting not to be changed after that) So you'll require extra memory for this as well. How much?? Again that depends on you concurrent applications and the frequency they return to read/write certain files.... Binaries used to have something like a sticky bit, so that the OS would know that these executables were in high demand, e.g. ls(1) on a multi-user server. And thus it would be kept a little longer in the file-cache.... Memory is cheap, I payed the other day something like < 100 Euro for 512Mb. I chose to use more memory versus buying faster processors. Next thing I would do is increase the number of spindels (aka disks), especially on the boxes that have losts of access from different clients for very different things. --WjW From owner-freebsd-performance@FreeBSD.ORG Sat Sep 11 09:30:37 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 59E4C16A4CE for ; Sat, 11 Sep 2004 09:30:37 +0000 (GMT) Received: from outmx004.isp.belgacom.be (outmx004.isp.belgacom.be [195.238.2.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id B838F43D58 for ; Sat, 11 Sep 2004 09:30:36 +0000 (GMT) (envelope-from geert@lori.mine.nu) Received: from outmx004.isp.belgacom.be (localhost [127.0.0.1]) with ESMTP id i8B9UWL9018573 for ; Sat, 11 Sep 2004 11:30:32 +0200 (envelope-from ) Received: from lori.mine.nu (19-141.244.81.adsl.skynet.be [81.244.141.19]) with ESMTP id i8B9UVHV018566; Sat, 11 Sep 2004 11:30:31 +0200 (envelope-from ) Received: by lori.mine.nu (Postfix, from userid 1000) id 1AC76659; Sat, 11 Sep 2004 11:30:31 +0200 (CEST) Date: Sat, 11 Sep 2004 11:30:30 +0200 From: Geert Hendrickx To: Willem Jan Withagen Message-ID: <20040911093030.GA40729@lori.mine.nu> References: <20040831133551.GA86660@lori.mine.nu> <4134B312.8030309@pacific.net.sg> <1093958674.680.2.camel@book> <20040831183908.GA87694@lori.mine.nu> <4141AA6A.2070802@withagen.nl> <20040910160051.GA24152@lori.mine.nu> <41422463.9090303@withagen.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <41422463.9090303@withagen.nl> User-Agent: Mutt/1.4.2i X-GPG-Key: http://lori.mine.nu/gnupgkey.asc X-GPG-Key-ID: 1024D/766C1E92 X-Accept-Language: nl,en cc: freebsd-performance@freebsd.org cc: oceanare@pacific.net.sg Subject: Re: spreading partitions over multiple drives X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Sep 2004 09:30:37 -0000 > Memory is cheap, I payed the other day something like < 100 Euro for > 512Mb. I chose to use more memory versus buying faster processors. Next > thing I would do is increase the number of spindels (aka disks), > especially on the boxes that have losts of access from different clients > for very different things. Instead of working out optimal partition layouts across the drives, I could also just use a Vinum RAID of course. I wouldn't opt for a RAID-0 configuration, as it seems too stupid to lose the entire contents of *two* disks when *one* of them fails. How does a RAID-1 compare to having only one disk, performancewise? When will Vinum use parallel reads? When one process is reading big chunks, or only when two or more processes are reading concurrently? Or is the Vinum RAID-1 algorithm not optimized for performance at all? Any thoughts/experiences on this? GH