From owner-freebsd-scsi Sat Jan 13 14:21:49 2001 Delivered-To: freebsd-scsi@freebsd.org Received: from h23.estsatnet (gw.estinc.com [216.216.240.51]) by hub.freebsd.org (Postfix) with ESMTP id 7EE1437B400 for ; Sat, 13 Jan 2001 14:21:30 -0800 (PST) Received: from localhost (IDENT:eric@localhost.localdomain [127.0.0.1]) by h23.estsatnet (8.9.3/8.9.3) with ESMTP id PAA06423 for ; Sat, 13 Jan 2001 15:23:13 -0700 Date: Sat, 13 Jan 2001 15:23:13 -0700 (MST) From: Eric Lee Green X-Sender: eric@h23.estsatnet To: freebsd-scsi@freebsd.org Subject: Why filemarks in sardpos? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-scsi@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This is under FreeBSD 4.2, as installed off of an ftp.freebsd.org mirror yesterday. In /usr/src/sys/cam/scsi/scsi_sa.c, in the sardpos routine, there is the following code: if (softc->flags & SA_FLAG_TAPE_WRITTEN) { error = sawritefilemarks(periph, 0, 0); if (error && error != EACCES) return (error); } I am using the GENERIC kernel. The above code makes the MTIOCRDSPOS ioctl have the following performance (kdump): 36104 bru 0.000025 CALL ioctl(0x5,MTIOCRDSPOS,0xbfbfecc4) 36104 bru 1.359931 RET ioctl 0 By contrast, when I comment that out, the MTIOCRDSPOS ioctl has the following performance: 391 bru 0.000025 CALL ioctl(0x5,MTIOCRDSPOS,0xbfbfed6c) 391 bru 0.001554 RET ioctl 0 Please note that this problem was noticed when I ported the QFA (Quick File Access) code in BRU 16.1 from Linux to FreeBSD (please do not ask me general questions about BRU, I will not know the answer -- I am EST's FreeBSD, tape drive, and tape changer person, I am not particularly knowledgable about BRU). QFA does a READ_POSITION during write time prior to writing each file (or each buffer, if we can fit multiple files into a BRU buffer) and saves that location data to a file or database. Then when it comes time to restore a single file, we tell BRU the filename and its tape position. The QFA code in BRU does a LOCATE to that location on the tape, grabs a buffer full of data (and perhaps additional bufferfulls if needed for a large file), and voila. Under Linux, this does not affect the performance of BRU during backups. Under FreeBSD *WITHOUT* the patch, BRU runs at snail's pace in QFA mode -- it is basically unusable, with a 1.35 second delay between each file written to tape, meaning we cannot build a QFA database this way (note that we can't just rely on how many buffers of data we've written out to tape -- we have no control over how the underlying OS does blocking, one BRU buffer could be multiple tape blocks). Under FreeBSD *WITH* the patch (that zaps sawritefilemarks), BRU runs at full speed. Note that the behavior of the Linux st.c driver corresponds to the behavior of my patched FreeBSD sa.c driver (where I zapped the sawritefilemarks). We have thoroughly tested the QFA code in BRU 16.1 with the st.c driver under Linux with every currently available enterprise-class tape drive and it correctly reports a logical tape position that is usable for restoring files. We have had over a dozen beta testers using BRU 16.1's QFA code under Linux for the past month without any reports of problems (and with much appreciation of being able to restore a lost file without having to read through the entire tape!). Here is my patch, BTW: -----------------------------snip--------------------------------------- --- sys/cam/scsi/scsi_sa.c.orig Sat Jan 13 14:25:44 2001 +++ sys/cam/scsi/scsi_sa.c Sat Jan 13 13:53:56 2001 @@ -3033,11 +3033,13 @@ * wary about trying to figure out the actual block location value * if data is in the tape drive buffer. */ +#ifdef STUPID_IDIOTIC_MORONIC_STUFF if (softc->flags & SA_FLAG_TAPE_WRITTEN) { error = sawritefilemarks(periph, 0, 0); if (error && error != EACCES) return (error); } +#endif ccb = cam_periph_getccb(periph, 1); scsi_read_position(&ccb->csio, 1, sadone, MSG_SIMPLE_Q_TAG, ------------------------------snip--------------------------------- Here is the corresponding code from Linux's st.c (this code is called directly from Linux's MTIOCPOS ioctl): memset (scmd, 0, 10); if ((STp->device)->scsi_level < SCSI_2) { scmd[0] = QFA_REQUEST_BLOCK; scmd[4] = 3; } else { scmd[0] = READ_POSITION; if (!logical && !STp->scsi2_logical) scmd[1] = 1; } SCpnt = st_do_scsi(NULL, STp, scmd, 20, STp->timeout, MAX_READY_RETRIES, TRUE); As I have mentioned, we have verified that the Linux behavior (simply issue the SSC READ_POSITION command *WITHOUT* a preceding FILEMARK command) works with all modern tape drives. If you want to see a subset of the tapedrives that we have tested this QFA code with, see http://www.linuxtapecert.org (please note that these are a severely limited subset of the drives we've actually tested). I have personally used: Tandberg SLR-50, Seagate DDS-3, Seagate DDS-4, HP DDS-3, HP DDS-4, Exabyte Mammoth, Exabyte Mammoth II, Ecrix VXA, Quantum DLT 4000, Quantum DLT 7000, Quantum DLT 8000, Seagate LTO, Sony AIT. On all of these drives, LOCATE properly finds the file using the READ POSITION number as reported without the preceding sawritefilemarks(). My conclusion is that the stuff commented out via #ifdef STUPID_IDIOTIC_MORONIC_STUFF is not necessary for proper operation of modern drives, and for drives that do need it, it slows down MTIOCRDSPOS to the point that it's useless anyhow (if we can't get the location of each file on tape because the call is so slow, why bother?) Final aside: It seems amazing that nobody else has ever run into this problem. Are we the only people who think that FreeBSD has potential for tape appliances in an enterprise backup setting? Are we the only people who have ever decided that FreeBSD is worthy of a port of an enterprise tape backup product? Surely our competitors aren't so lame that they would ignore QFA for quick restores? Oh well. At least we can still support backing up a FreeBSD machine to a Linux or Solaris box via BRU Professional, even though we can't officially support using FreeBSD as a BP tape server until we have an "official" version of FreeBSD with the capability of doing so (I will try to get an "unsupported" version of the BRU Professional tape server onto the CD, along with the patch, but obviously we can't officially support a product that requires a kernel patch in order to properly operate). -- Eric Lee Green eric@estinc.com Software Engineer "The BRU Guys" Enhanced Software Technologies, Inc. http://www.estinc.com/ (602) 470-1115 voice (602) 470-1116 fax To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-scsi" in the body of the message