Date: Thu, 07 Dec 2006 15:21:46 +0000 From: Dieter <freebsd@sopwith.solgatos.com> To: Kris Kennaway <kris@obsecurity.org> Cc: freebsd-questions@freebsd.org Subject: Re: processes not getting fair share of available disk I/O (was: Re: TCP parameters and interpreting tcpdump output ) Message-ID: <200612072321.XAA23385@sopwith.solgatos.com> In-Reply-To: Your message of "Thu, 23 Nov 2006 14:20:11 EST." <20061123192011.GA81772@xor.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > > > > hw.ata.wc=3D3D3D0 > > > > > ^^^^^^^^^^^ > > > > > "Make my hard drive go reeeeally slow please (just in case I crash)= > " :) > > > >=3D20 > > > > Slower, yes, but not *that* slow. > > > >=3D20 > > > > Normal ls : 0.032 second. Two processes using same disk, multiply by= > two, > > > > so 0.064 second. Maybe the multiplier is more than 2, call it 10x, so > > > > 0.32 second. But I'm seeing a factor of over 9100x. > > >=20 > > > Humour me and turn it back on, then see what happens. > >=20 > > Where is the knob to turn the write cache on/off on a per-drive basis > > in FreeBSD? I can do this in NetBSD, but the only knob I can find in > > FreeBSD affects all drives, and requires a reboot. > > Yes, I think you need to do it globally at boot time. > > > Humour me and read the Subject line. The ls does not get its fair share > > of disk I/O. > >=20 > > Both times are with the disk's write cache in write-through mode. > > I'm not comparing times with the write cache in different modes. > > I'm comparing ls by itself against ls competing with cp. > > Your cp is going to be running synchronously, i.e. spend a lot of time > waiting on the disk to perform the writes. This may well be the cause > of your problem. Once we have established whether or not it is the > cause, we can proceed to whether this behaviour can be improved. I submitted PR 106340 asking for a way to control the disk write cache on a per disk basis like NetBSD can. Meanwhile, I added a PATA via USB disk, which judging from the write speed, appears to be immune from hw.ata.wc=0. So I now have a disk which has the write cache on, is connected via a different controller, and thus uses a different device driver. I still see the same problems. Writing to one disk *significantly* slows down writing to another disk. Even if one process is at normal default priority and the other is running at rtprio 5. Regardless of which process uses the USB disk and which uses the direct-to-chipset disk. Even if the rtprio 5 process only needs a very small fraction of the disk bandwidth, it still gets slowed down to the point that data is lost. My current SWAG is that writing to a disk requires some spl/mutex/lock that is global across all disks on the system. And this spl/mutex/lock is a bottleneck.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200612072321.XAA23385>