Date: Fri, 18 Jan 2013 16:48:10 -0800 From: Dieter BSD <dieterbsd@gmail.com> To: freebsd-hackers@freebsd.org Cc: gibbs@FreeBSD.org, scottl@FreeBSD.org, mjacob@FreeBSD.org Subject: Re: IBM blade server abysmal disk write performances Message-ID: <CAA3ZYrBToG6DP8qDH7JOSGozzsx8nzEZQiBF6dg=S_Q%2BKi7qpw@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Scott writes: > If I had my way, the WC would be off, everyone would be using SAS, > and anyone who enabled SATA WC or complained about I/O slowness > would be forced into Siberian salt mines for the remainder of their lives. Actually, If you are running SAS, having SATA WC on or off wouldn't matter, it would be SCSI's WC you'd care about. :-) >> The bigger problem is that >> FreeBSD does not support queuing on all controllers that support it. >> Not something that admins can fix, and inexcusable for an OS that >> claims to care about performance. > > You keep saying this, but I'm unclear on what you mean. Can you > explain? For most applications you need the write cache to be off. Having the write cache off is fine as long as you have queuing. But with the write cache off, if you don't have queuing, performance sucks. Like getting only 6% of the performance you should be getting. Some of the early SATA controllers didn't have NCQ. Knowing that queuing was very important, I made sure to choose a mainboard with NCQ, giving up other useful features to get it. But FreeBSD does not support NCQ on the nforce4-ultra's SATA controllers. Even the sad joke of an OS Linux has had NCQ on nforce4 since Oct 2006. But Linux is such crap it is unusable. Linux is slowly improving, but I don't expect to live long enough to see it become usable. Seriously. I've tried it several times but I have completely given up on it. Anyway, even after all these years the supposedly performance oriented FreeBSD still does not support NCQ on nforce4, which isn't some obscure chip. they sold a lot them. I've added 3 additional SATA controllers on expansion cards, and FreeBSD supports NCQ on them, so the slow controllers limited by PCIe-x1 have much better write performance than the much faster controllers in the chipset with all the bandwidth they need. I can't add more controllers, there aren't any free slots. The nforce will remain in service for years, aside from the monetary cost, silicon has a huge amount of environmental cost: embedded energy, water, pollution, etc. And there are a lot of them. Wojciech writes: >> That is incorrect. A UPS reduces the risk, but does not eliminate it. > > nothing eliminate all risks. Turning the write cache off eliminates the risk of having the write cache on. Yes you can still lose data for other reasons. Backups are still a good idea. >> But for most applications, you must have the write cache off, >> and you need queuing (e.g. TCQ or NCQ) for performance. If >> you have queuing, there is no need to turn the write cache >> on. > > did you tested the above claim? i have SATA drives everywhere, all in ahci > mode, all with NCQ active. Yes, turn the write cache off and NCQ will give you the performance. As long as you have queuing you can have the best of both worlds. Which is why Karim's problem is so odd. Driver says there is queuing, but performance (1 write per rev) looks exactly like there is no queuing. Maybe there is something else that causes only 1 write per rev but I don't know what that might be. Peter writes: > Apart from continuous whinging and whining on mailing lists, what have > you done to add support for queuing? Submitted PR, it was closed without being fixed. Looked at code, but Greek to me, even though I have successfully modified a BSD based device driver in the past giving major performance improvement. If I were a C-level exec of a Fortune 500 company I'd just hire some device driver wizard.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAA3ZYrBToG6DP8qDH7JOSGozzsx8nzEZQiBF6dg=S_Q%2BKi7qpw>