Date: Wed, 31 Mar 1999 23:28:04 -0600 From: David Kelly <dkelly@hiwaay.net> To: "Justin T. Gibbs" <gibbs@narnia.plutotech.com> Cc: scsi@FreeBSD.org Subject: Re: cache and tape drives Message-ID: <199904010528.XAA47587@nospam.hiwaay.net> In-Reply-To: Message from "Justin T. Gibbs" <gibbs@narnia.plutotech.com> of "Wed, 31 Mar 1999 21:26:33 MST." <199904010426.VAA43947@narnia.plutotech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
"Justin T. Gibbs" writes:
> > Have to admit CAM is over my head as I haven't found my answer. Haven't
> > been sniffing around with a kernel debugger either. Suspect the ccb
> > LINK and FLAG bits control the process blocking.
>
> Not at all. Processes block on writes to all 'raw' devices. The
That explains something. Scanning my mail archives says Bruce Evans
told me that on 3/10. Am suffering from overload of data.
> reason for this is simple. To avoid a copy of your data, the kernel
> blocks your process, wires the data buffer in your process into memory
> so it cannot be swapped out, and performs the I/O to the device. Only
> once the I/O has completed can the call to write(2) complete to ensure
> the proper semantics of the system call (e.g. you can reuse your buffer
> as soon as write returns).
>
> What you are seeing in your application is a classic latency problem.
> You are transfering data between two high latency devices, so your
> total throughput will suffer if you are only using a single process:
>
> while (more_data) {
> read from tape [block for a long while...]
> write to tape [block for a long while...]
> }
>
> The write side likely has much less latency than the read side at
> least until the write buffer in the device is filled. DDS-3 drives
> do have a write buffer and the sa driver will make use of it. On
> the read side, the tape driver must block until the data is available.
> If the latency of transfering the read data into the write buffer
> of the target tape device is large enough, you won't be able to
> stream your reads, and performance will suffer.
Last month when I knocked off 180 tapes in roughly lots of 10 of the
same tape my procedure was to create a single file image of the tape on
HD, a tar image, then to use "tcopy -s 10240 file.tar /dev/rsa0", but in
a shell script where I kicked off one tcopy per tape drive, all tcopys
reading the same file. Writing to DDS-1 tapes I saw 700k/sec and steady
but for some reason when writing to DDS-3 tapes (same drives) it was
only 550k/sec and widely varying. It could have been the tapes as some
wrote and verified OK but the drive LED was flashing, complaining about
error rates.
> So how do you solve the problem? For FreeBSD, you'll need to use
> multiple processes to perform the reads and writes with some kind
> of fast buffer in the middle.
I tried team and dd and didn't see a thing change.
Have brought up this subject before when I believed I was simulating my
4 tape drive thruput problems when rc5des was running on my home
machine. With rc5des running at its default nice (20), using dd to write
10k blocks from /dev/zero to /dev/rsa0 varies between 150k and 200k/sec
on this machine. Without rc5des (but X running) its 360k to 380k. Bruce
Evans provided a patch to kern_synch.c. Couldn't tell any change in
thruput but there was some change to the "feel" of the system at the
keyboard. Was not able to try that patch on the 4 tape machine.
Others have suggested bumping the kernel HZ up from 100 to maybe 1000.
But from my understanding of things that would simply be a bandaid and
no more as a blocked process is supposed to be able to push a running
process aside when it becomes unblocked. Or something like that.
Shortly I should be getting another system to play with. Again. One with
4 or 8 tape drives. And not have to inaccurately simulate the problem
with rc5des.
--
David Kelly N4HHE, dkelly@nospam.hiwaay.net
=====================================================================
The human mind ordinarily operates at only ten percent of its
capacity -- the rest is overhead for the operating system.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-scsi" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199904010528.XAA47587>