From owner-freebsd-stable@FreeBSD.ORG Thu Nov 28 21:03:17 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95FEBE2C for ; Thu, 28 Nov 2013 21:03:17 +0000 (UTC) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 508011503 for ; Thu, 28 Nov 2013 21:03:17 +0000 (UTC) Received: from compute2.internal (compute2.nyi.mail.srv.osa [10.202.2.42]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 9254C20BE2 for ; Thu, 28 Nov 2013 16:03:09 -0500 (EST) Received: from frontend2 ([10.202.2.161]) by compute2.internal (MEProxy); Thu, 28 Nov 2013 16:03:09 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= date:from:to:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=mesmtp; bh=QThTkFL8n0lllt7SGcM/FTZ/TiI=; b=MiNPTERshIROsezicKKocwE02yHj uTIFxnDrgCRwptXPZlYFrKKCFafyNAEdtLhHGR5sQiHKB2eQpt9gFiJ0h1JCu3nG PNAFWozTybvoGNQ8yzwOyXxZAXtyShJixumQJP1EkDNtXtNsZGJV57d5DCLlt50D eJEgFc4zqs96Wis= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=date:from:to:subject:message-id :in-reply-to:references:mime-version:content-type :content-transfer-encoding; s=smtpout; bh=QThTkFL8n0lllt7SGcM/FT Z/TiI=; b=TYnxFor0i9byJjhrROw2sEpiPVnA8xNovpJ0A3yFKhYRxcSlYwz3Vf Smk1hzgEyOfhW3R0U8ogZLaCOzqkpfkSG4/OX3AKuTDNkMopMC8qO2rsHQ/4JbMy 9Sb7xfcyl0WRDkwOsJTskVAuQWXtq4dbuKtlY+hPVQemWrOINVzo0= X-Sasl-enc: oKohJPfx8QNkfsQQkmqPHShQvLhaTD8kkFf3X9HH79Gc 1385672589 Received: from harmony.localnet.edu (unknown [141.87.213.68]) by mail.messagingengine.com (Postfix) with ESMTPA id 23481680135 for ; Thu, 28 Nov 2013 16:03:09 -0500 (EST) Date: Thu, 28 Nov 2013 22:03:05 +0100 From: Schaich Alonso To: freebsd-stable@freebsd.org Subject: Re: gmirror: writes are faster than reads Message-Id: <20131128220305.e715adb95b16f494224052f5@fastmail.fm> In-Reply-To: <5297ABD5.5060504@gmail.com> References: <5297ABD5.5060504@gmail.com> X-Mailer: Sylpheed 3.3.0 (GTK+ 2.24.19; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Nov 2013 21:03:17 -0000 On Thu, 28 Nov 2013 21:47:17 +0100 Szalai Andr=C3=A1s wrote: > Hi Guys, >=20 > Has somebody encountered (significantly) different read/write speeds > when using gmirror? >=20 > I have 2xWD WD30EFRX RED drives which are configured as follows: >=20 > $ gmirror status > Name Status Components > mirror/root COMPLETE ada0p2 (ACTIVE) > ada1p2 (ACTIVE) > mirror/data COMPLETE ada0p4 (ACTIVE) > ada1p4 (ACTIVE) >=20 > mirror/root is mounted as the root fs (UFS2). >=20 > Doing write: >=20 > $ time dd if=3D/dev/zero of=3D/IMAGE bs=3D1024k count=3D`expr 4 \* 1024` > 4096+0 records in > 4096+0 records out > 4294967296 bytes transferred in 29.326044 secs (146455733 bytes/sec) >=20 > Doing read: >=20 > $ time dd if=3D/IMAGE of=3D/dev/null bs=3D1024k count=3D`expr 4 \* 1024` > 4096+0 records in > 4096+0 records out > 4294967296 bytes transferred in 48.821649 secs (87972598 bytes/sec) >=20 > As you can see, read is much slower than write (87 vs 146 MB/s). Why? >=20 > Any help would be appreciated. >=20 > Best regards, > Andrew >=20 > PS: Partition layout (partitions are 4k aligned): >=20 > $ gpart show ada0 ada1 > =3D> 34 5860533101 ada0 GPT (2.7T) > 34 6 - free - (3.0k) > 40 1024 1 freebsd-boot (512k) > 1064 16777216 2 freebsd-ufs (8.0G) > 16778280 16777216 3 freebsd-swap (8.0G) > 33555496 5826977632 4 freebsd-ufs (2.7T) > 5860533128 7 - free - (3.5k) >=20 > =3D> 34 5860533101 ada1 GPT (2.7T) > 34 6 - free - (3.0k) > 40 1024 1 freebsd-boot (512k) > 1064 16777216 2 freebsd-ufs (8.0G) > 16778280 16777216 3 freebsd-swap (8.0G) > 33555496 5826977632 4 freebsd-ufs (2.7T) > 5860533128 7 - free - (3.5k) Modern HDDs have both Command Queuing and Excessive Cache Memory. Using them a spindle disk can cache multiple write requests and do them all in one revolution. While multiple read requests can also be done at once chances are the on-disk cache is not usefull (because the requested data is only resident if it was accessed short before, and then it's also availible in the kernel's larger Filesystem/GEOM caches which reside in main memory and were consulted prior to disk accesses) and the GEOM layer might not have issued them yet. IIRC the UFS subsystem will perform no read requests small= er than 512kB by default, which means it does some readahead just in case the issuing application wants to read more data soon - however you have used re= ad blocks which are exact multiples of 512kB, so there is no gain in this. readahead is the buzzword for tuning large sequencial reads, and I had thou= ght there was a sysctl for it, though I can't find it now. Alonso