From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 19 00:48:11 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1799612F; Sat, 19 Jan 2013 00:48:11 +0000 (UTC) (envelope-from dieterbsd@gmail.com) Received: from mail-ie0-f180.google.com (mail-ie0-f180.google.com [209.85.223.180]) by mx1.freebsd.org (Postfix) with ESMTP id B5BAAEA; Sat, 19 Jan 2013 00:48:10 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id c10so7004799ieb.25 for ; Fri, 18 Jan 2013 16:48:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to:cc :content-type; bh=bNZTILOqSDrmlZVUlbXQriwObkgWPL614/z0YITuooo=; b=WCuD/zIIG96uPmWpMADJEj6hOnI4yRgQR5tcygU84nKdNh5eFBJhFZKNywSHSpPfnN 27ZLMJV1GQHOg5pzz2H56+CyFYsoRiUTf+2JceKPbqM66Vt/4BRsmqvvbkUDZnYoYu4u S6UhNYpvOsEf6Gd7rXwQIBuVrceiOf8/qgJW4gObvcbkge0ehbm931scQsUxADj3iezm PTdN+q878URYEJAyXtTERAq3RyEHuU/zWjyjf2XDHFBBEKYFo6OhdKuWerQnXBldI5xc 9pXAVOX7c5gVLZVsfb2QJeDydxDRCq6DPkrB7BP9J06jQhVzDt1W2bPX9eOLQ/0BxbM4 KbSw== MIME-Version: 1.0 X-Received: by 10.42.153.70 with SMTP id l6mr7395923icw.50.1358556490296; Fri, 18 Jan 2013 16:48:10 -0800 (PST) Received: by 10.64.107.196 with HTTP; Fri, 18 Jan 2013 16:48:10 -0800 (PST) Date: Fri, 18 Jan 2013 16:48:10 -0800 Message-ID: Subject: Re: IBM blade server abysmal disk write performances From: Dieter BSD To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: gibbs@FreeBSD.org, scottl@FreeBSD.org, mjacob@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Jan 2013 00:48:11 -0000 Scott writes: > If I had my way, the WC would be off, everyone would be using SAS, > and anyone who enabled SATA WC or complained about I/O slowness > would be forced into Siberian salt mines for the remainder of their lives. Actually, If you are running SAS, having SATA WC on or off wouldn't matter, it would be SCSI's WC you'd care about. :-) >> The bigger problem is that >> FreeBSD does not support queuing on all controllers that support it. >> Not something that admins can fix, and inexcusable for an OS that >> claims to care about performance. > > You keep saying this, but I'm unclear on what you mean. Can you > explain? For most applications you need the write cache to be off. Having the write cache off is fine as long as you have queuing. But with the write cache off, if you don't have queuing, performance sucks. Like getting only 6% of the performance you should be getting. Some of the early SATA controllers didn't have NCQ. Knowing that queuing was very important, I made sure to choose a mainboard with NCQ, giving up other useful features to get it. But FreeBSD does not support NCQ on the nforce4-ultra's SATA controllers. Even the sad joke of an OS Linux has had NCQ on nforce4 since Oct 2006. But Linux is such crap it is unusable. Linux is slowly improving, but I don't expect to live long enough to see it become usable. Seriously. I've tried it several times but I have completely given up on it. Anyway, even after all these years the supposedly performance oriented FreeBSD still does not support NCQ on nforce4, which isn't some obscure chip. they sold a lot them. I've added 3 additional SATA controllers on expansion cards, and FreeBSD supports NCQ on them, so the slow controllers limited by PCIe-x1 have much better write performance than the much faster controllers in the chipset with all the bandwidth they need. I can't add more controllers, there aren't any free slots. The nforce will remain in service for years, aside from the monetary cost, silicon has a huge amount of environmental cost: embedded energy, water, pollution, etc. And there are a lot of them. Wojciech writes: >> That is incorrect. A UPS reduces the risk, but does not eliminate it. > > nothing eliminate all risks. Turning the write cache off eliminates the risk of having the write cache on. Yes you can still lose data for other reasons. Backups are still a good idea. >> But for most applications, you must have the write cache off, >> and you need queuing (e.g. TCQ or NCQ) for performance. If >> you have queuing, there is no need to turn the write cache >> on. > > did you tested the above claim? i have SATA drives everywhere, all in ahci > mode, all with NCQ active. Yes, turn the write cache off and NCQ will give you the performance. As long as you have queuing you can have the best of both worlds. Which is why Karim's problem is so odd. Driver says there is queuing, but performance (1 write per rev) looks exactly like there is no queuing. Maybe there is something else that causes only 1 write per rev but I don't know what that might be. Peter writes: > Apart from continuous whinging and whining on mailing lists, what have > you done to add support for queuing? Submitted PR, it was closed without being fixed. Looked at code, but Greek to me, even though I have successfully modified a BSD based device driver in the past giving major performance improvement. If I were a C-level exec of a Fortune 500 company I'd just hire some device driver wizard.