From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 16 01:26:49 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6E475105 for ; Wed, 16 Jan 2013 01:26:49 +0000 (UTC) (envelope-from fodillemlinkarim@gmail.com) Received: from mail-ia0-x232.google.com (mail-ia0-x232.google.com [IPv6:2607:f8b0:4001:c02::232]) by mx1.freebsd.org (Postfix) with ESMTP id 41131BF7 for ; Wed, 16 Jan 2013 01:26:49 +0000 (UTC) Received: by mail-ia0-f178.google.com with SMTP id y26so64137iab.37 for ; Tue, 15 Jan 2013 17:26:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=I0nTBrs4T1x33kcTIljdIl5Y6a/TNVfPZOg2S4fWiqo=; b=TO11UsQyNZAFpcf8sUJdIGLio1mDEeGVBUVwmINc3GRaXzv3GP8aDDNpRwiwVabRn1 TILBq8O+BA3Gv3NdgrG1aS/l1QOiqDe3QBMgaRhoyE5z9sUj/HZEewKuYjkfo3hWLzrI wpy7kU1K9tTggQKeHw603Sl2Fy+ouGps3U8Zv4vGOpnsN5QHNHPnciKhC6O3KthScYiI rC9Qg0xwb/UKu3zHzC5uz0EpCagf1XlF9wMgvB43h6U1CWYgPy6FA6nd+eYK2U0A7R0a XCcrQc+sQ3DCHHf0lMziFE8nFUDjJg6/OG/ES3VAOpztq49iZEk+S8osIZMkZMjjwwvY 71NQ== X-Received: by 10.43.7.7 with SMTP id om7mr100142icb.25.1358299608736; Tue, 15 Jan 2013 17:26:48 -0800 (PST) Received: from [10.0.0.131] ([24.225.136.71]) by mx.google.com with ESMTPS id iw5sm3315952igc.5.2013.01.15.17.26.46 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 15 Jan 2013 17:26:47 -0800 (PST) Message-ID: <50F601D2.3090009@gmail.com> Date: Tue, 15 Jan 2013 20:26:42 -0500 From: Karim Fodil-Lemelin User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Wojciech Puchar Subject: Re: IBM blade server abysmal disk write performances References: <50F5BC08.1060700@gmail.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Wed, 16 Jan 2013 03:51:41 +0000 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jan 2013 01:26:49 -0000 On 15/01/2013 4:54 PM, Wojciech Puchar wrote: >> >> # dd if=/dev/zero of=foo count=1 bs=10240000 >> 1+0 records in >> 1+0 records out >> 10240000 bytes transferred in 19.579077 secs (523007 bytes/sec) >> > you write to file not device, so it will be clustered anyway by FreeBSD. > > 128kB by default, more if you put options MAXPHYS=... in kernel config > and recompile. > > Even with hard drive write cache disabled, it should about one write > per revolution but seems to do 4 writes per second. > > so probably it is not that but much worse failure. > > Did you rest read speed? > > dd if=/dev/disk of=/dev/null bs=512 > > dd if=/dev/disk of=/dev/null bs=4k > > dd if=/dev/disk of=/dev/null bs=128k > > ? > I'll do the read test as well but if I recall correctly it seemed pretty decent. It is quite obvious that something is awfully slow on SAS drives, whatever it is and regardless of OS comparison. We swapped the SAS drives for SATA and we're seeing much higher speeds. Basically on par with what we were expecting (roughly 300 to 400 times faster then what we see with SAS...). I find it strange that diskinfo reports those transfer rates: Transfer rates: outside: 102400 kbytes in 0.685483 sec = 149384 kbytes/sec middle: 102400 kbytes in 0.747424 sec = 137004 kbytes/sec inside: 102400 kbytes in 1.051036 sec = 97428 kbytes/sec Yet we get only a tiny fraction of those (it takes 20 seconds to transfer 10MB!) when using dd. I also doubt its dd's behavior since how can we explain the performance going up with SATA when doing the same test? Unfortunately, we'll have to move on soon and we're about to write off SAS and use SATA instead. Thanks, Karim.