Date: Mon, 2 Mar 2015 10:28:46 -0700 From: "Kenneth D. Merry" <ken@FreeBSD.ORG> To: Dan Langille <dan@langille.org> Cc: current@freebsd.org, scsi@freebsd.org Subject: Re: sa(4) driver changes available for test Message-ID: <20150302172846.GB87055@mithlond.kdm.org> In-Reply-To: <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <D1C8E304-6DE7-4A03-A350-64E097EC4804@langille.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote:
>
> > On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry <ken@FreeBSD.ORG> wrote:
> >
> > On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote:
> >>
> >>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry <ken@FreeBSD.ORG> wrote:
> >>>
> >>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote:
> >>>>
> >>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry <ken@FreeBSD.ORG> wrote:
> >>>>>
> >>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote:
> >>>>>>
> >>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry <ken@freebsd.org> wrote:
> >>>>>>>
> >>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote:
> >>>>>>>>
> >>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry <ken@freebsd.org> wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I have a fairly large set of changes to the sa(4) driver and mt(1) driver
> >>>>>>>>> that I'm planning to commit in the near future.
> >>>>>>>>>
> >>>>>>>>> A description of the changes is here and below in this message.
> >>>>>>>>>
> >>>>>>>>> If you have tape hardware and the inclination, I'd appreciate testing and
> >>>>>>>>> feedback.
> >>>>>>>>
> >>>>>>>> I have a DLT 8000 and an SDLT 220.
> >>>>>>>>
> >>>>>>>> I don't have anything running current, but I have a spare machine which I could use for testing.
> >>>>>>>>
> >>>>>>>> Do you see any value is tests with that hardware? I'd be testing it via Bacula.
> >>>>>>>>
> >>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula committer.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your trying it out
> >>>>>>> would be helpful if you have the time.
> >>>>>>>
> >>>>>>> In looking at the manuals for both the SDLT 220 and the DLT 8000, they both
> >>>>>>> claim to support long position information for the SCSI READ POSITION
> >>>>>>> command.
> >>>>>>>
> >>>>>>> You can see what I'm talking about by doing:
> >>>>>>>
> >>>>>>> mt eod
> >>>>>>> mt status
> >>>>>>>
> >>>>>>> On my DDS-4 tape drive, this shows:
> >>>>>>>
> >>>>>>> # mt -f /dev/nsa3 status
> >>>>>>> Drive: sa3: <SEAGATE DAT 06240-XXX 8071> Serial Number: HJ00YWY
> >>>>>>> ---------------------------------
> >>>>>>> Mode Density Blocksize bpi Compression
> >>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled (DCLZ)
> >>>>>>> ---------------------------------
> >>>>>>> Current Driver State: at rest.
> >>>>>>> ---------------------------------
> >>>>>>> Partition: 0 Calc File Number: -1 Calc Record Number: -1
> >>>>>>> Residual: 0 Reported File Number: -1 Reported Record Number: -1
> >>>>>>> Flags: None
> >>>>>>>
> >>>>>>> But on an LTO-5, which will give long position information, I get:
> >>>>>>>
> >>>>>>> [root@doc ~]# mt status
> >>>>>>> Drive: sa0: <IBM ULTRIUM-HH5 E4J1>
> >>>>>>> ---------------------------------
> >>>>>>> Mode Density Blocksize bpi Compression
> >>>>>>> Current: 0x58:LTO-5 variable 384607 enabled (0x1)
> >>>>>>> ---------------------------------
> >>>>>>> Current Driver State: at rest.
> >>>>>>> ---------------------------------
> >>>>>>> Partition: 0 Calc File Number: 2 Calc Record Number: -1
> >>>>>>> Residual: 0 Reported File Number: 2 Reported Record Number: 32373
> >>>>>>> Flags: None
> >>>>>>>
> >>>>>>> That, in combination with the changes I made to the position information
> >>>>>>> code in the driver, mean that even the old MTIOCGET ioctl should return an
> >>>>>>> accurate file number at end of data. e.g., on the LTO-5:
> >>>>>>>
> >>>>>>> [root@doc ~]# mt ostatus
> >>>>>>> Mode Density Blocksize bpi Compression
> >>>>>>> Current: 0x58:LTO-5 variable 384607 0x1
> >>>>>>> ---------available modes---------
> >>>>>>> 0: 0x58:LTO-5 variable 384607 0x1
> >>>>>>> 1: 0x58:LTO-5 variable 384607 0x1
> >>>>>>> 2: 0x58:LTO-5 variable 384607 0x1
> >>>>>>> 3: 0x58:LTO-5 variable 384607 0x1
> >>>>>>> ---------------------------------
> >>>>>>> Current Driver State: at rest.
> >>>>>>> ---------------------------------
> >>>>>>> File Number: 2 Record Number: -1 Residual Count -1
> >>>>>>>
> >>>>>>> So the thing to try, in addition to just making sure that Bacula continues
> >>>>>>> to work properly, is to try setting this for the tape drive in
> >>>>>>> bacula-sd.conf:
> >>>>>>>
> >>>>>>> Hardware End of Medium = yes
> >>>>>>>
> >>>>>>> It looks like the Bacula tape program (btape) has a test mode, and it would
> >>>>>>> be good to run through the tests on one of the tape drives and see whether
> >>>>>>> they work, and whether the results are different before and after the
> >>>>>>> changes. I'm not sure how to enable the test mode.
> >>>>>>
> >>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf
> >>>>>>
> >>>>>> Device {
> >>>>>> Name = DLT
> >>>>>> Description = "QUANTUM DLT7000 1624"
> >>>>>> Media Type = DLT
> >>>>>> Archive Device = /dev/nsa1
> >>>>>>
> >>>>>> Autochanger = YES
> >>>>>> Drive Index = 0
> >>>>>>
> >>>>>> Offline On Unmount = no
> >>>>>> Hardware End of Medium = yes
> >>>>>> BSF at EOM = yes
> >>>>>> Backward Space Record = no
> >>>>>> Fast Forward Space File = no
> >>>>>> TWO EOF = yes
> >>>>>> }
> >>>>>>
> >>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) has a btape test on this same model.
> >>>>>>
> >>>>>> Here's the test I ran tonight:
> >>>>>>
> >>>>>> [root@cuppy:/usr/home/dan] # btape -c /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1
> >>>>>> Tape block granularity is 1024 bytes.
> >>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing.
> >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
> >>>>>> *test
> >>>>>>
> >>>>>> === Write, rewind, and re-read test ===
> >>>>>>
> >>>>>> I'm going to write 10000 records and an EOF
> >>>>>> then write 10000 records and an EOF, then rewind,
> >>>>>> and re-read the data to verify that it is correct.
> >>>>>>
> >>>>>> This is an *essential* feature ...
> >>>>>>
> >>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes.
> >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes.
> >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:1210-0 Rewind OK.
> >>>>>> 10000 blocks re-read correctly.
> >>>>>> Got EOF on tape.
> >>>>>> 10000 blocks re-read correctly.
> >>>>>> === Test Succeeded. End Write, rewind, and re-read test ===
> >>>>>>
> >>>>>> btape: btape.c:1277-0 Block position test
> >>>>>> btape: btape.c:1289-0 Rewind OK.
> >>>>>> Reposition to file:block 0:4
> >>>>>> Block 5 re-read correctly.
> >>>>>> Reposition to file:block 0:200
> >>>>>> Block 201 re-read correctly.
> >>>>>> Reposition to file:block 0:9999
> >>>>>> Block 10000 re-read correctly.
> >>>>>> Reposition to file:block 1:0
> >>>>>> Block 10001 re-read correctly.
> >>>>>> Reposition to file:block 1:600
> >>>>>> Block 10601 re-read correctly.
> >>>>>> Reposition to file:block 1:9999
> >>>>>> Block 20000 re-read correctly.
> >>>>>> === Test Succeeded. End Write, rewind, and re-read test ===
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> === Append files test ===
> >>>>>>
> >>>>>> This test is essential to Bacula.
> >>>>>>
> >>>>>> I'm going to write one record in file 0,
> >>>>>> two records in file 1,
> >>>>>> and three records in file 2
> >>>>>>
> >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
> >>>>>> btape: btape.c:1909-0 Wrote block to device.
> >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
> >>>>>> btape: btape.c:1909-0 Wrote block to device.
> >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
> >>>>>> btape: btape.c:1909-0 Wrote block to device.
> >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
> >>>>>> btape: btape.c:1909-0 Wrote block to device.
> >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
> >>>>>> btape: btape.c:1909-0 Wrote block to device.
> >>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes.
> >>>>>> btape: btape.c:1909-0 Wrote block to device.
> >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK
> >>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1)
> >>>>>> btape: btape.c:1420-0 Now moving to end of medium.
> >>>>>
> >>>>> This is the critical piece. The test moves the tape to the end of the
> >>>>> medium. With hardware position information, you can tell what filemark
> >>>>> you're on. Without it, you can't.
> >>>>>
> >>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on "DLT" (/dev/nsa1). ERR=No error: 0.
> >>>>>> We should be in file 3. I am at file 0. This is NOT correct!!!!
> >>>>>>
> >>>>>> Append test failed. Attempting again.
> >>>>>> Setting "Hardware End of Medium = no
> >>>>>> and "Fast Forward Space File = no
> >>>>>> and retrying append test.
> >>>>>
> >>>>> This is not surprsing, given that the drive doesn't support long read
> >>>>> position data. (It's a SCSI-2 device.) So that means that Bacula will
> >>>>> need to do it manually.
> >>>>
> >>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 but that
> >>>> tape library is hooked up to a different computer and was doing backups today.
> >>>
> >>> So, here is one thing that we can try to see whether these drives support
> >>> long position information, even though they only claim to be SCSI-2. If
> >>> they do, we can potentially add a quirk (or autodetection) to enable it.
> >>> The code currently doesn't bother asking drives that claim to be SCSI-2
> >>> for long position information. (Because that feature was added in the
> >>> SSC spec, which came after SCSI-2.)
> >>>
> >>> Issue a READ POSITION with the short form specified:
> >>>
> >>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd
> >>>
> >>> Issue a READ POSITION with the vendor-specific block numbers:
> >>>
> >>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd
> >>>
> >>> Issue a READ POSITION with the long form data:
> >>>
> >>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd
> >>>
> >>> If it supports the last one, then I can put a quirk (or autodetection) in
> >>> the driver and Bacula will get the hardware filemarks. You should try this
> >>> on your SDLT as well. It may well support it.
> >>
> >> Sadly, no:
> >>
> >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd
> >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> >> 00000010 00 00 00 00 |....|
> >> 00000014
> >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd
> >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> >> 00000010 00 00 00 00 |....|
> >> 00000014
> >> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd
> >> camcontrol: error sending command
> >> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 00
> >> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error
> >> (pass2:ahc0:0:2:0): SCSI status: Check Condition
> >> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB)
> >> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid
> >> [root@cuppy:~] #
> >
> > Okay. Not too surprising I suppose.
> >
> >> The SDLT server is on 9.3 though:
> >>
> >> [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd
> >> camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed
> >> cam_lookup_pass: No such file or directory
> >> cam_lookup_pass: either the pass driver isn't in your kernel
> >> cam_lookup_pass: or sa1 doesn't exist
> >> [root@knew:/usr/home/dan] # uname -a
> >> FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 #0: Tue Feb 24 21:28:03 UTC 2015 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64
> >> [root@knew:/usr/home/dan] #
> >>
> >>
> >> It took me a while to figure that out... there is no sa1 on *this* system.
> >>
> >> But, my SDLT:
> >>
> >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd
> >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> >> 00000010 00 00 00 00 |....|
> >> 00000014
> >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd
> >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> >> 00000010 00 00 00 00 |....|
> >> 00000014
> >> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd
> >> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> >> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> >> 00000020
> >> [root@knew:/usr/home/dan] #
> >
> > Just to confirm, can you send the output of:
> >
> > camcontrol inquiry sa0 -v
> >
> > I want to make certain it reports that it is SCSI-2. If so, I'll change
> > the check in the driver to try asking for long position information on
> > SCSI-2 devices. If it fails, it'll fall back to the regular method.
>
> [dan@knew:~] $ sudo camcontrol inquiry sa0 -v
> pass10: <COMPAQ SuperDLT1 5F5F> Removable Sequential Access SCSI-2 device
> pass10: Serial Number CXB46H0716
> pass10: 80.000MB/s transfers (40.000MHz, offset 31, 16bit)
> [dan@knew:~] $
Okay. Doing the check doesn't cause any problems on my collection of old
tape drives, so I'll go ahead and enable checking on SCSI-2 devices.
The primary thing is just making sure it doesn't cause tape drive to choke
if we request long position information. It doesn't appear to be an
issue. If we do run into one, we can put in a quirk.
Ken
--
Kenneth Merry
ken@FreeBSD.ORG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150302172846.GB87055>