From owner-freebsd-stable@FreeBSD.ORG Mon Jan 25 07:00:59 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6626D1065676; Mon, 25 Jan 2010 07:00:59 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f226.google.com (mail-fx0-f226.google.com [209.85.220.226]) by mx1.freebsd.org (Postfix) with ESMTP id 8CAE68FC29; Mon, 25 Jan 2010 07:00:58 +0000 (UTC) Received: by fxm26 with SMTP id 26so689533fxm.13 for ; Sun, 24 Jan 2010 23:00:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=p6eu0eZZ0xKVt/XSKpZZv3bpWEX+BO2tnASnZOTfEdI=; b=akjh+F4rYocKFdx5jCLmq8q9J8ADZajKCHIAu1a551RSIutwk4Q4n02bBMA8y4efPE 0NOHAkM5FZLoAsU/doxkAVrOsMw7vMzONTtNfJUpJxbDrwMgXRF81iThvdkZaSWhV2gp Uij9axdqnqu6L29u73Cp334YacXUlQYp05Hs4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=HWOJBXaf7RbmnT473CpKL2PGOPv+IIvS8mh7560RQ529+PleQ4ENllw317EOMeiRX9 ss03Rv2p7sbIGfRHci6LNMDiUt2FHzcZH/M+jhRIMiinj9qiCNrHV3Ql+vVG7jAnsBHz yvDBYpB/9h+ouYgOu2Sz1zaiyDI09j3uHXUzI= Received: by 10.223.17.155 with SMTP id s27mr6428939faa.13.1264402857645; Sun, 24 Jan 2010 23:00:57 -0800 (PST) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 15sm2542816fxm.6.2010.01.24.23.00.56 (version=SSLv3 cipher=RC4-MD5); Sun, 24 Jan 2010 23:00:57 -0800 (PST) Sender: Alexander Motin Message-ID: <4B5D41A6.5080203@FreeBSD.org> Date: Mon, 25 Jan 2010 09:00:54 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Dan Naumov References: <883b2dc51001240905r4cfbf830i3b9b400969ac261b@mail.gmail.com> <1264368182.00211075.1264355402@10.7.7.3> <4B5CC167.5010604@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Jason Edwards , FreeBSD-STABLE Mailing List , freebsd-questions@freebsd.org Subject: Re: 8.0-RELEASE/amd64 - full ZFS install - low read and write disk performance X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jan 2010 07:00:59 -0000 Dan Naumov wrote: > On Mon, Jan 25, 2010 at 2:14 AM, Dan Naumov wrote: >> On Sun, Jan 24, 2010 at 11:53 PM, Alexander Motin wrote: >>> Dan Naumov wrote: >>>> This works out to 1GB in 36,2 seconds / 28,2mb/s in the first test and >>>> 4GB in 143.8 seconds / 28,4mb/s and somewhat consistent with the >>>> bonnie results. It also sadly seems to confirm the very slow speed :( >>>> The disks are attached to a 4-port Sil3124 controller and again, my >>>> Windows benchmarks showing 65mb/s+ were done on exact same machine, >>>> with same disks attached to the same controller. Only difference was >>>> that in Windows the disks weren't in a mirror configuration but were >>>> tested individually. I do understand that a mirror setup offers >>>> roughly the same write speed as individual disk, while the read speed >>>> usually varies from "equal to individual disk speed" to "nearly the >>>> throughput of both disks combined" depending on the implementation, >>>> but there is no obvious reason I am seeing why my setup offers both >>>> read and write speeds roughly 1/3 to 1/2 of what the individual disks >>>> are capable of. Dmesg shows: >>>> >>>> atapci0: port 0x1000-0x100f mem >>>> 0x90108000-0x9010807f,0x90100000-0x90107fff irq 21 at device 0.0 on >>>> pci4 >>>> ad8: 1907729MB at ata4-master SATA300 >>>> ad10: 1907729MB at ata5-master SATA300 >>> 8.0-RELEASE, and especially 8-STABLE provide alternative, much more >>> functional driver for this controller, named siis(4). If your SiI3124 >>> card installed into proper bus (PCI-X or PCIe x4/x8), it can be really >>> fast (up to 1GB/s was measured). >>> >>> -- >>> Alexander Motin >> Sadly, it seems that utilizing the new siis driver doesn't do much good: >> >> Before utilizing siis: >> >> iozone -s 4096M -r 512 -i0 -i1 >> random >> random bkwd record stride >> KB reclen write rewrite read reread read >> write read rewrite read fwrite frewrite fread freread >> 4194304 512 28796 28766 51610 50695 >> >> After enabling siis in loader.conf (and ensuring the disks show up as ada): >> >> iozone -s 4096M -r 512 -i0 -i1 >> >> random >> random bkwd record stride >> KB reclen write rewrite read reread read >> write read rewrite read fwrite frewrite fread freread >> 4194304 512 28781 28897 47214 50540 > > Just to add to the numbers above, exact same benchmark, on 1 disk > (detached 2nd disk from the mirror) while using the siis driver: > > random > random bkwd record stride > KB reclen write rewrite read reread read > write read rewrite read fwrite frewrite fread freread > 4194304 512 57760 56371 68867 74047 If both parts of mirror uses same controller, it doubles it's bus traffic. That may reduce bandwidth twice. The main benefit of siis(4) is a command queuing. You should receive more benefits on multithread random I/O. -- Alexander Motin