From owner-freebsd-geom@FreeBSD.ORG Fri Dec 29 12:05:21 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5816716A512; Fri, 29 Dec 2006 12:05:21 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from jengal.datamax.bg (jengal.datamax.bg [82.103.104.21]) by mx1.freebsd.org (Postfix) with ESMTP id 26A2713C48D; Fri, 29 Dec 2006 12:05:19 +0000 (UTC) (envelope-from vd@datamax.bg) Received: from qlovarnika.bg.datamax (qlovarnika.bg.datamax [192.168.10.2]) by jengal.datamax.bg (Postfix) with SMTP id 0E00DB84C; Fri, 29 Dec 2006 14:05:18 +0200 (EET) Received: (nullmailer pid 15882 invoked by uid 1002); Fri, 29 Dec 2006 12:05:18 -0000 Date: Fri, 29 Dec 2006 14:05:18 +0200 From: Vasil Dimov To: freebsd-geom@freebsd.org Message-ID: <20061229120517.GA12877@qlovarnika.bg.datamax> References: <20061228171858.GA11296@qlovarnika.bg.datamax> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="tKW2IUtsqtDRztdT" Content-Disposition: inline In-Reply-To: <20061228171858.GA11296@qlovarnika.bg.datamax> Subject: Re: gstripe performance scaling with many disks X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: vd@FreeBSD.org List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2006 12:05:21 -0000 --tKW2IUtsqtDRztdT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, Thank you very much for your answers! Here is what further tests showed: First of all I switched from dd to raidtest. I had to tune raidtest in order to use it, see ports/107311. I am testing only reading with 8 concurrent processes (raidtest test -n 8). 2 disks: 2103646 b/s (exp 2107584), avg disk load: 1051823 ( 99.8%) 3 disks: 3134534 b/s (exp 3161376), avg disk load: 1044844 ( 99.1%) 4 disks: 4153974 b/s (exp 4215168), avg disk load: 1038493 ( 98.5%) 5 disks: 5199917 b/s (exp 5268960), avg disk load: 1039983 ( 98.6%) 6 disks: 6141678 b/s (exp 6322752), avg disk load: 1023613 ( 97.1%) 7 disks: 7193116 b/s (exp 7376544), avg disk load: 1027588 ( 97.5%) 8 disks: 8219609 b/s (exp 8430336), avg disk load: 1027451 ( 97.5%) 9 disks: 9080762 b/s (exp 9484128), avg disk load: 1008973 ( 95.7%) 10 disks: 10241349 b/s (exp 10537920), avg disk load: 1024134 ( 97.1%) 11 disks: 11077983 b/s (exp 11591712), avg disk load: 1007089 ( 95.5%) 12 disks: 11851009 b/s (exp 12645504), avg disk load: 987584 ( 93.7%) 13 disks: 12663548 b/s (exp 13699296), avg disk load: 974119 ( 92.4%) 14 disks: 13821213 b/s (exp 14753088), avg disk load: 987229 ( 93.6%) 15 disks: 14283895 b/s (exp 15806880), avg disk load: 952259 ( 90.3%) 16 disks: 15057168 b/s (exp 16860672), avg disk load: 941073 ( 89.3%) 17 disks: 16171889 b/s (exp 17914464), avg disk load: 951287 ( 90.2%) It shows the same tendency as with dd(1). Changing vfs.read_max from 8 to 32 does not produce different results. This are gstat screen shots during the test: 1 drive: dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 8 12 12 1003 548.1 0 0 0.0 96.6| ggate100 8 drives: dT: 0.501s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 6 116 116 1028 63.4 0 0 0.0 97.6| ggate100 2 118 118 1080 50.6 0 0 0.0 102.0| ggate101 4 114 114 1042 45.9 0 0 0.0 100.8| ggate102 4 110 110 982 39.3 0 0 0.0 96.0| ggate103 4 116 116 1027 43.7 0 0 0.0 98.6| ggate104 6 116 116 1056 58.4 0 0 0.0 101.6| ggate105 8 124 124 1029 60.0 0 0 0.0 98.0| ggate107 7 122 122 1051 61.9 0 0 0.0 106.6| ggate106 8 130 130 8230 62.8 0 0 0.0 99.6| stripe/stest 17 drives: dT: 0.563s w: 0.500s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 5 185 178 848 18.9 0 0 0.0 88.8| ggate100 4 183 176 831 19.8 0 0 0.0 87.9| ggate101 4 181 174 822 18.8 0 0 0.0 87.0| ggate102 4 165 165 801 17.7 0 0 0.0 86.6| ggate103 2 167 167 820 17.3 0 0 0.0 85.0| ggate104 1 176 176 840 18.8 0 0 0.0 86.4| ggate105 4 167 167 812 23.5 0 0 0.0 85.5| ggate107 3 167 167 802 19.9 0 0 0.0 85.9| ggate108 1 172 172 841 19.9 0 0 0.0 86.5| ggate109 2 170 170 852 28.5 0 0 0.0 87.4| ggate110 4 172 172 847 21.4 0 0 0.0 87.1| ggate111 4 174 174 838 22.3 0 0 0.0 87.0| ggate112 5 163 163 789 16.2 0 0 0.0 86.8| ggate113 4 162 162 802 15.7 0 0 0.0 86.4| ggate114 3 162 162 823 17.0 0 0 0.0 86.2| ggate115 3 170 170 825 18.2 0 0 0.0 86.6| ggate116 5 169 169 810 25.7 0 0 0.0 86.1| ggate106 8 245 238 13573 27.5 0 0 0.0 90.1| stripe/stest I have not tried geom_cache... --=20 Vasil Dimov gro.DSBeerF@dv % If the code and the comments disagree, then both are probably wrong. -- Norm Schryer --tKW2IUtsqtDRztdT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- iD8DBQFFlQR9Fw6SP/bBpCARAoBIAJ4srn9bRpyLZyy8lJxgAIqJUinDBQCg2o07 1qCl6ArTT4kcuA014CGGZ2o= =oSEe -----END PGP SIGNATURE----- --tKW2IUtsqtDRztdT--