Date: Wed, 13 Dec 2006 16:37:42 +0000 From: Dieter <freebsd@sopwith.solgatos.com> To: freebsd-questions@freebsd.org Subject: Re: processes not getting fair share of available disk I/O Message-ID: <200612140037.AAA14550@sopwith.solgatos.com> In-Reply-To: Your message of "Wed, 13 Dec 2006 18:27:26 EST." <20061213232726.GA61149@xor.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> > Is Giant the only mutex/lock that could be a bottleneck across disks? > > The only one I can think of that is generic. One would have to do > more extensive profiling and diagnosis to try and figure out what is > wrong with your system. Suggestions of what to look at would be welcome. > The only explanation that seems to fit is that it's something to do > with your particular hardware (i.e. driver issue), since it's > certainly not a problem on general configurations. > > I know that many people have bad things to say about nforce chipsets, > although I dont know if your particular problem has been reported > before. Could APIC have anything to do with this? It is currently turned off in firmware. Today I experimented with vfs.hirunningspace. If I crank it up, I get better total write speed with multiple drives doing dd from /dev/zero to files on disks. But it doesn't help my real applications, and in fact appears to hurt them. I have also discovered that my program that reads from Ethernet and does non-blocking writes to stdout which get redirected to a disk file isn't benefiting as much as it should from the large circular buffer. The buffer should be enough to hold 4.5-5 minutes worth of data, and is mlock(2)'d into memory. I can read from four disks at once without hurting the Ethernet transfer. But a dd from /dev/zero writing to a different disk that takes less than 3 seconds interferes with the Ethernet transfer. The fcntl(1, O_NONBLOCK); isn't failing, and the write() isn't returning -1/EAGAIN.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200612140037.AAA14550>