From owner-freebsd-hardware@FreeBSD.ORG Fri Mar 15 10:27:51 2013 Return-Path: Delivered-To: hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9EF8A674 for ; Fri, 15 Mar 2013 10:27:51 +0000 (UTC) (envelope-from amdmi3@amdmi3.ru) Received: from smtp.timeweb.ru (smtp.timeweb.ru [92.53.116.43]) by mx1.freebsd.org (Postfix) with ESMTP id 5FD99126 for ; Fri, 15 Mar 2013 10:27:51 +0000 (UTC) Received: from [213.148.20.85] (helo=hive.panopticon) by smtp.timeweb.ru with esmtpsa (TLS1.0:DHE_RSA_CAMELLIA_256_CBC_SHA1:256) (Exim 4.80) (envelope-from ) id 1UGQQb-0006yt-II for hardware@FreeBSD.org; Fri, 15 Mar 2013 12:55:53 +0400 Received: from hades.panopticon (hades.panopticon [192.168.0.32]) by hive.panopticon (Postfix) with ESMTP id 01C78B84D for ; Fri, 15 Mar 2013 12:55:52 +0400 (MSK) Received: by hades.panopticon (Postfix, from userid 1000) id 695A2AD2; Fri, 15 Mar 2013 12:55:52 +0400 (MSK) Date: Fri, 15 Mar 2013 12:55:52 +0400 From: Dmitry Marakasov To: hardware@FreeBSD.org Subject: Uneven disk throughput distribution Message-ID: <20130315085552.GD99263@hades.panopticon> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Mar 2013 10:27:51 -0000 Hi! Not sure if it's FreeBSD problem, but I thought it won't hurt to write about it. I'm playing with a set of new 3TB Seagate ST3000VX000-1CU166 hard disks. Each disk does 160-180MB/s write throughtput with `dd if=/dev/zero of=/dev/adaX bs=10m`, but when all disks are written to simultaneously, throughput distribution is uneven: ada1: 174032158720 bytes transferred in 1486.316119 secs (117089599 bytes/sec) ada2: 174199930880 bytes transferred in 1487.372522 secs (117119234 bytes/sec) ada3: 172805324800 bytes transferred in 1488.324386 secs (116107299 bytes/sec) ada4: 169376481280 bytes transferred in 1489.353551 secs (113724831 bytes/sec) ada5: 157642915840 bytes transferred in 1490.324518 secs (105777577 bytes/sec) ada6: 112449290240 bytes transferred in 1491.338958 secs (75401564 bytes/sec) I guess that summary performance drop is to be expected due to bus/controller throughput cap, but uneven distribution is strange, and if it may be fixed, it may improve performance of large raid arrays. The system is FreeBSD 9.0 amd64. PS. I've tried the same thing on Debian live system. Similar pattern of throughput drop towards the last disk, however the difference is even worse: 142, 140, 136, 130, 117, 49 MB/s -- Dmitry Marakasov . 55B5 0596 FF1E 8D84 5F56 9510 D35A 80DD F9D2 F77D amdmi3@amdmi3.ru ..: jabber: amdmi3@jabber.ru http://www.amdmi3.ru