From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 16 03:54:58 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 443AD411 for ; Wed, 16 Jan 2013 03:54:58 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id EDF86377 for ; Wed, 16 Jan 2013 03:54:57 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id r0G3ss5I065909 for ; Tue, 15 Jan 2013 20:54:54 -0700 (MST) (envelope-from ian@FreeBSD.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r0G3spjW004215; Tue, 15 Jan 2013 20:54:51 -0700 (MST) (envelope-from ian@FreeBSD.org) Subject: Re: IBM blade server abysmal disk write performances From: Ian Lepore To: Karim Fodil-Lemelin In-Reply-To: <50F5BC08.1060700@gmail.com> References: <50F5BC08.1060700@gmail.com> Content-Type: text/plain; charset="us-ascii" Date: Tue, 15 Jan 2013 20:54:51 -0700 Message-ID: <1358308491.32417.147.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jan 2013 03:54:58 -0000 On Tue, 2013-01-15 at 15:28 -0500, Karim Fodil-Lemelin wrote: > On 15/01/2013 3:03 PM, Dieter BSD wrote: > > Disabling the disks's write cache is *required* for data integrity. > > One op per rev means write caching is disabled and no queueing. > > But dmesg claims "Command Queueing enabled", so you should be > getting > > more than one op per rev, and writes should be fast. > > Is this dd to the raw drive, to a filesystem? (FFS? ZFS? other?) > > Are you running compression, encryption, or some other feature > > that might slow things down? Also, try dd with a larger block size, > > like bs=1m. > Hi, > > Thanks to everyone that answered so far. Here is a follow up. dd to > the > raw drive and no compression/encryption or some other features, just > a > naive boot off a live 9.1 CD then dd (see below). The following > results > have been gathered on the FreeBSD 9.1 system: You say dd with a raw drive, but as several people have pointed out, linux dd doesn't go directly to the drive by default. It looks like you can make it do so with the "direct" option, which should make it behave the same as freebsd's dd behaves by default (I think, I'm no linux expert). For example, using a usb thumb drive: th2 # dd if=/dev/sdb4 of=/dev/null count=100 100+0 records in 100+0 records out 51200 bytes (51 kB) copied, 0.0142396 s, 3.6 MB/s th2 # dd if=/dev/sdb4 of=/dev/null count=100 iflag=direct 100+0 records in 100+0 records out 51200 bytes (51 kB) copied, 0.0628582 s, 815 kB/s Hmm, just before hitting send I saw your other response that SAS drives behave badly, SATA are fine. That does seem to point away from dd behavior. It might still be interesting to see if the direct flag on linux drops performance into the same horrible range as freebsd with SAS. -- Ian