From owner-freebsd-scsi Wed Mar 31 21:28:36 1999 Delivered-To: freebsd-scsi@freebsd.org Received: from mail.HiWAAY.net (fly.HiWAAY.net [208.147.154.56]) by hub.freebsd.org (Postfix) with ESMTP id EEAF914FBA for ; Wed, 31 Mar 1999 21:28:31 -0800 (PST) (envelope-from dkelly@nospam.hiwaay.net) Received: from nospam.hiwaay.net (tnt8-216-180-15-157.dialup.HiWAAY.net [216.180.15.157]) by mail.HiWAAY.net (8.9.1a/8.9.0) with ESMTP id XAA28856; Wed, 31 Mar 1999 23:28:09 -0600 (CST) Received: from nospam.hiwaay.net (nospam.hiwaay.net [127.0.0.1]) by nospam.hiwaay.net (8.9.2/8.9.2) with ESMTP id XAA47587; Wed, 31 Mar 1999 23:28:05 -0600 (CST) (envelope-from dkelly@nospam.hiwaay.net) Message-Id: <199904010528.XAA47587@nospam.hiwaay.net> X-Mailer: exmh version 2.0.2 2/24/98 To: "Justin T. Gibbs" Cc: scsi@FreeBSD.org From: David Kelly Subject: Re: cache and tape drives In-reply-to: Message from "Justin T. Gibbs" of "Wed, 31 Mar 1999 21:26:33 MST." <199904010426.VAA43947@narnia.plutotech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 31 Mar 1999 23:28:04 -0600 Sender: owner-freebsd-scsi@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org "Justin T. Gibbs" writes: > > Have to admit CAM is over my head as I haven't found my answer. Haven't > > been sniffing around with a kernel debugger either. Suspect the ccb > > LINK and FLAG bits control the process blocking. > > Not at all. Processes block on writes to all 'raw' devices. The That explains something. Scanning my mail archives says Bruce Evans told me that on 3/10. Am suffering from overload of data. > reason for this is simple. To avoid a copy of your data, the kernel > blocks your process, wires the data buffer in your process into memory > so it cannot be swapped out, and performs the I/O to the device. Only > once the I/O has completed can the call to write(2) complete to ensure > the proper semantics of the system call (e.g. you can reuse your buffer > as soon as write returns). > > What you are seeing in your application is a classic latency problem. > You are transfering data between two high latency devices, so your > total throughput will suffer if you are only using a single process: > > while (more_data) { > read from tape [block for a long while...] > write to tape [block for a long while...] > } > > The write side likely has much less latency than the read side at > least until the write buffer in the device is filled. DDS-3 drives > do have a write buffer and the sa driver will make use of it. On > the read side, the tape driver must block until the data is available. > If the latency of transfering the read data into the write buffer > of the target tape device is large enough, you won't be able to > stream your reads, and performance will suffer. Last month when I knocked off 180 tapes in roughly lots of 10 of the same tape my procedure was to create a single file image of the tape on HD, a tar image, then to use "tcopy -s 10240 file.tar /dev/rsa0", but in a shell script where I kicked off one tcopy per tape drive, all tcopys reading the same file. Writing to DDS-1 tapes I saw 700k/sec and steady but for some reason when writing to DDS-3 tapes (same drives) it was only 550k/sec and widely varying. It could have been the tapes as some wrote and verified OK but the drive LED was flashing, complaining about error rates. > So how do you solve the problem? For FreeBSD, you'll need to use > multiple processes to perform the reads and writes with some kind > of fast buffer in the middle. I tried team and dd and didn't see a thing change. Have brought up this subject before when I believed I was simulating my 4 tape drive thruput problems when rc5des was running on my home machine. With rc5des running at its default nice (20), using dd to write 10k blocks from /dev/zero to /dev/rsa0 varies between 150k and 200k/sec on this machine. Without rc5des (but X running) its 360k to 380k. Bruce Evans provided a patch to kern_synch.c. Couldn't tell any change in thruput but there was some change to the "feel" of the system at the keyboard. Was not able to try that patch on the 4 tape machine. Others have suggested bumping the kernel HZ up from 100 to maybe 1000. But from my understanding of things that would simply be a bandaid and no more as a blocked process is supposed to be able to push a running process aside when it becomes unblocked. Or something like that. Shortly I should be getting another system to play with. Again. One with 4 or 8 tape drives. And not have to inaccurately simulate the problem with rc5des. -- David Kelly N4HHE, dkelly@nospam.hiwaay.net ===================================================================== The human mind ordinarily operates at only ten percent of its capacity -- the rest is overhead for the operating system. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-scsi" in the body of the message