From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 18 17:26:13 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8B368C7A for ; Fri, 18 Jan 2013 17:26:13 +0000 (UTC) (envelope-from scott4long@yahoo.com) Received: from nm20.bullet.mail.ne1.yahoo.com (nm20.bullet.mail.ne1.yahoo.com [98.138.90.83]) by mx1.freebsd.org (Postfix) with ESMTP id 49EFDA89 for ; Fri, 18 Jan 2013 17:26:13 +0000 (UTC) Received: from [98.138.90.57] by nm20.bullet.mail.ne1.yahoo.com with NNFMP; 18 Jan 2013 17:22:59 -0000 Received: from [98.138.89.169] by tm10.bullet.mail.ne1.yahoo.com with NNFMP; 18 Jan 2013 17:22:59 -0000 Received: from [127.0.0.1] by omp1025.mail.ne1.yahoo.com with NNFMP; 18 Jan 2013 17:22:59 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 32151.48370.bm@omp1025.mail.ne1.yahoo.com Received: (qmail 74387 invoked by uid 60001); 18 Jan 2013 17:22:58 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1358529778; bh=D/8jjl4OZi/A5XTFk2tJcBNEnRF9W+i8s6MiAhROvz4=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=eJF29s8YISW1MA/ZxpNbzbl4Zf2GFtQCmPaO5n7GD2wCOfO8SxYT2tSkZqnvdnk1IsUzuqXYuCj4DyMTSEq6hSALwEaqzwRa36n3ZLtkVkfLZRqcbre6Z3fmMVqEQedGFAP8SdifFrriPQPrBcSvsdjwqyn3SM/J4eVdRNxmWks= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=cUNngRCbtFyJMepHAmQTNN2tyRpoyCmR0Qpp9pFvFa9uJP89y3c/X/IF5LTSMT14ZR/iN2rmFJ89KKYgj14Og0SaImuDsPR+jj5wAoJQza60QzmDsdx+/KkAkIUtVmkJIETI//BYM89U1lmkfwHgtCChTsksiOFbyE7uMsfIiAA=; X-YMail-OSG: 6sazS4gVM1neG4XLSZOSIol1Mg9j.VyzEBGfoTL7orPrun4 Jdhw0QdacN8P5F_gYESY6_BLziaxNFQzejM1774FStXPtxFP0OjC6khOvvl4 TtuOr6opUrD7u1kvIOf9nvAPe1xpAd0LO4tnBqVaMETdT2gNMoFNekLPURX4 tU4YR.IFXPTadKczqmzBlnWG08ZUL74x5Vx8B2DAoYrTVeA8IwiGTglgxLRt uiUkyv8ShTrzwSFE.ILTJ9Y7IWANEEmdq415vANwwQMhjvZoZwnKHn73OLQR yatfjHYw8bGo4ujzQqY0eUs6QVWRz68JTAsfdjakp5N4zfyRnVeQiwNafJkd Z8wLBnnXrMHDCYUFPHqRLQFjkRt8YSMN7Qys1aHkNY_l1Z2_IMs5RRgOusBy JK02Oq2kl5VWdjBcG5zH_BiV03qGPt3pDRa83jeAQdgRoxM6tzJDOwcMr.Ml c648CeUjE4WGeVHMiMGKRnlgolTMFfES3a0lZE.ODn2d6a3UUdVS8Xmfxt1z G2Hh.tLwZHeYhmtO.l9Z_KzLd Received: from [69.53.237.126] by web120304.mail.ne1.yahoo.com via HTTP; Fri, 18 Jan 2013 09:22:58 PST X-Rocket-MIMEInfo: 001.001, VHJ5IGFkZGluZyB0aGUgZm9sbG93aW5nIHRvIC9ib290L2xvYWRlci5jb25mIGFuZCByZWJvb3Q6Cgpody5tcHQuZW5hYmxlX3NhdGFfd2M9MQoKVGhlIGRlZmF1bHQgdmFsdWUsIC0xLCBpbnN0cnVjdHMgdGhlIGRyaXZlciB0byBsZWF2ZSB0aGUgU1RBIGRyaXZlcyBhdCB0aGVpciBjb25maWd1cmF0aW9uIGRlZmF1bHQuIMKgT2Z0ZW4gdGltZXMgdGhpcyBtZWFucyB0aGF0IHRoZSBNUFQgQklPUyB3aWxsIHR1cm4gb2ZmIHRoZSB3cml0ZSBjYWNoZSBvbiBldmVyeSBzeXN0ZW0gYm9vdCBzZXF1ZW5jZS4gwqABMAEBAQE- X-Mailer: YahooMailWebService/0.8.130.494 References: Message-ID: <1358529778.71931.YahooMailNeo@web120304.mail.ne1.yahoo.com> Date: Fri, 18 Jan 2013 09:22:58 -0800 (PST) From: Scott Long Subject: Re: IBM blade server abysmal disk write performances To: Dieter BSD , "freebsd-hackers@freebsd.org" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Fri, 18 Jan 2013 17:50:25 +0000 Cc: "gibbs@FreeBSD.org" , "scottl@FreeBSD.org" , "mjacob@FreeBSD.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Scott Long List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2013 17:26:13 -0000 Try adding the following to /boot/loader.conf and reboot:=0A=0Ahw.mpt.enabl= e_sata_wc=3D1=0A=0AThe default value, -1, instructs the driver to leave the= STA drives at their configuration default. =A0Often times this means that = the MPT BIOS will turn off the write cache on every system boot sequence. = =A0IT DOES THIS FOR A GOOD REASON! =A0An enabled write cache is counter to = data reliability. =A0Yes, it helps make benchmarks look really good, and it= 's acceptable if your data can be safely thrown away (for example, you're j= ust caching from a slower source, and the cache can be rebuilt if it gets c= orrupted). =A0And yes, Linux has many tricks to make this benchmark look re= ally good. =A0The tricks range from buffering the raw device to having 'dd'= recognize the requested task and short-circuit the process of going to /de= v/null or pulling from /dev/zero. =A0I can't tell you how bogus these tests= are and how completely irrelevant they are in predicting actual workload p= erformance. =A0But, I'm not going to stop anyone from trying, so give the a= bove tunable a try and let me know how it works.=0A=0ABtw, I'm not subscribed to the hackers = mailing list, so please redistribute this email as needed.=0A=0AScott=0A=0A= =0A=0A=0A>________________________________=0A> From: Dieter BSD =0A>To: freebsd-hackers@freebsd.org =0A>Cc: mjacob@FreeBSD.org; g= ibbs@FreeBSD.org; scottl@FreeBSD.org =0A>Sent: Thursday, January 17, 2013 9= :03 PM=0A>Subject: Re: IBM blade server abysmal disk write performances=0A>= =0A>> I am thinking that something fancy in that SAS drive is=0A>> not bei= ng handled correctly by the FreeBSD driver.=0A>=0A>I think so too, and I th= ink the something fancy is "tagged command queuing".=0A>The driver prints "= da0: Command Queueing enabled" and yet your SAS drive=0A>is only getting 1 = write per rev, and queuing should get you more than that.=0A>Your SATA driv= e is getting the expected performance, which means that NCQ=0A>must be work= ing.=0A>=0A>> Please let me know if there is anything you would like me to = run on the=0A>> BSD 9.1 system to help diagnose this issue?=0A>=0A>Looking = at the mpt driver, a verbose boot may give more info.=0A>Looks like you can= set a "debug" device hint, but I don't=0A>see any documentation on what to= set it to.=0A>=0A>I think it is time to ask the driver wizards why TCQ isn= 't working,=0A>so I'm cc-ing the authors listed on the mpt man page.=0A>=0A= >=0A>