From owner-freebsd-fs@FreeBSD.ORG Mon Oct 9 19:38:28 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1689716A403 for ; Mon, 9 Oct 2006 19:38:28 +0000 (UTC) (envelope-from bde@zeta.org.au) Received: from mailout2.pacific.net.au (mailout2-3.pacific.net.au [61.8.2.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4AF1443D45 for ; Mon, 9 Oct 2006 19:38:27 +0000 (GMT) (envelope-from bde@zeta.org.au) Received: from mailproxy2.pacific.net.au (mailproxy2.pacific.net.au [61.8.2.163]) by mailout2.pacific.net.au (Postfix) with ESMTP id 325996E125; Tue, 10 Oct 2006 05:38:25 +1000 (EST) Received: from epsplex.bde.org (katana.zip.com.au [61.8.7.246]) by mailproxy2.pacific.net.au (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id k99JcKSo014207; Tue, 10 Oct 2006 05:38:21 +1000 Date: Tue, 10 Oct 2006 05:37:33 +1000 (EST) From: Bruce Evans X-X-Sender: bde@epsplex.bde.org To: "Fluffles.net" In-Reply-To: <45297DA2.4000509@fluffles.net> Message-ID: <20061010051216.G814@epsplex.bde.org> References: <45297DA2.4000509@fluffles.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@FreeBSD.org, Kris Kennaway Subject: Re: 2 bonnies can stop disk activity permanently X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Oct 2006 19:38:28 -0000 On Mon, 9 Oct 2006, Fluffles.net wrote: > I'm the "veronica" Arne mentioned in the freebsd-fs mailinglist. > Regarding the effectiveness of a higher blocksize, these are my findings: > > areca RAID5 (8x da, 128KB stripe, default newfs, NCQ enabled) > -------Sequential Output-------- ---Sequential Input-- > --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- > --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU > /sec %CPU > ARC8xR5 8480 119973 91.3 247178 58.6 67862 17.5 90426 86.9 172490 24.0 > 120.7 0.5 > > areca RAID5 (8x da, 128KB stripe, 64KB blocksize newfs, NCQ enabled) > -------Sequential Output-------- ---Sequential Input-- > --Random-- > -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- > --Seeks--- > Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU > /sec %CPU > ARC8xR5 8480 128920 97.8 265920 58.9 116787 31.0 103261 97.8 392970 > 53.8 119.8 0.6 > > As you can see, the block read increased from ~172MB/s to ~392MB/s, > quite significant increase. Also the reqrite speed increased from > ~67MB/s to ~116MB/s. > > Ofcourse these tests are on a brand clean filesystem, which might not > tally with real-life crowded filesystems. But at least there is much > ... This is a bit surprising. FreeBSD is supposed to cluster the i/o so that (especially for large files on new file systems) almost all i/o is done in blocks of size 64K or 128K. I suspect the problems are that the 64K-block i/o is usually perfectly misaligned unless the fs itself has 64K-blocks and the fs's partition starts on a 64K-block boundary, and that some hardware or firmware (mainly RAIDs) want the blocks to be aligned. I'm not very familiar with RAIDs but think it would take a fairly advanced/expensive one to reblock all the i/at so that the alignment doesn't matter. It would take more advanced/complicated clustering code or better buffering code than FreeBSD has to do the reblocking at the clustering or buffering level. Perhaps even 64K-blocks are too small with your RAID's stripe size of 128K. Bruce