From owner-freebsd-geom@FreeBSD.ORG Thu Dec 28 18:25:22 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 38FCF16A417 for ; Thu, 28 Dec 2006 18:25:16 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id D1D5813C475 for ; Thu, 28 Dec 2006 18:25:15 +0000 (UTC) (envelope-from gcubfg-freebsd-geom@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1Gzzw3-0000il-HL for freebsd-geom@freebsd.org; Thu, 28 Dec 2006 19:24:59 +0100 Received: from 89-172-61-42.adsl.net.t-com.hr ([89.172.61.42]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:24:59 +0100 Received: from ivoras by 89-172-61-42.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 28 Dec 2006 19:24:59 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-geom@freebsd.org From: Ivan Voras Date: Thu, 28 Dec 2006 19:24:45 +0100 Lines: 49 Message-ID: References: <20061228171858.GA11296@qlovarnika.bg.datamax> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig9A7097DADAAF2593CDBF794C" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 89-172-61-42.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) In-Reply-To: <20061228171858.GA11296@qlovarnika.bg.datamax> X-Enigmail-Version: 0.94.1.2 Sender: news Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2006 18:25:22 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig9A7097DADAAF2593CDBF794C Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Vasil Dimov wrote: > Can someone explain this? > The tendency is for performace drop when increasing the number of disks= > in a stripe but there are some local peaks/extremums when using 8, 11 > and 16 disks. I'll take a shot at this: Since maximum kernel reads are still limited to 128 KB/s, by adding more drives you're making individual requests shorter. I.e. with one drive, it gets 128 KB requests, with two, each gets 64 KB, with 16, each gets 8 KB. So network & kernel latency becomes visible. AFAIK there's unofficial (still?) GEOM_CACHE class which tries to get around this by requesting & caching 128K from each drive. Search the lists, it's mentioned somewhere. >=20 > Yes, I have read > http://lists.freebsd.org/pipermail/freebsd-geom/2006-November/001705.ht= ml >=20 > kern.geom.stripe.fast is set to 1. While you're playing with this, you could set vfs.read_max to 32 or higher and see if it helps. --------------enig9A7097DADAAF2593CDBF794C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFlAvzldnAQVacBcgRApOtAKDPqJ/HJfmf6M6jvctjitjzfkHVsgCgv59d ftyB+tS4eELnv1k02rWxpTM= =sgLO -----END PGP SIGNATURE----- --------------enig9A7097DADAAF2593CDBF794C--