From owner-freebsd-current@FreeBSD.ORG Mon Mar 2 19:07:38 2015 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 83B58CA7; Mon, 2 Mar 2015 19:07:38 +0000 (UTC) Received: from clavin1.langille.org (clavin.langille.org [162.208.116.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "clavin.langille.org", Issuer "StartCom Class 2 Primary Intermediate Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 53479653; Mon, 2 Mar 2015 19:07:37 +0000 (UTC) Received: from (clavin1.int.langille.org (clavin1.int.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) with ESMTPSA id 211FB6FEC ; Mon, 2 Mar 2015 19:07:36 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: sa(4) driver changes available for test From: Dan Langille In-Reply-To: <20150302172846.GB87055@mithlond.kdm.org> Date: Mon, 2 Mar 2015 14:07:35 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <12EED94D-CC9D-44C4-91C2-62C8A51B839B@langille.org> References: <20150214003232.GA63990@mithlond.kdm.org> <7CA52DF3-E073-4F50-BE4E-01C51CCDF2C7@langille.org> <20150217183645.GA30947@mithlond.kdm.org> <303C48AC-0DA8-447C-8DE7-0A1DD5DD7351@langille.org> <20150302003150.GB71528@mithlond.kdm.org> <20150302022957.GB73433@mithlond.kdm.org> <0F17CDDB-4CE9-4881-B948-6B9BEDEB6455@langille.org> <20150302163146.GA85515@mithlond.kdm.org> <85E2DDA0-4A2B-4316-A318-3706B3A05C11@langille.org> <20150302172846.GB87055@mithlond.kdm.org> To: "Kenneth D. Merry" X-Mailer: Apple Mail (2.2070.6) Cc: current@freebsd.org, scsi@freebsd.org X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Mar 2015 19:07:38 -0000 > On Mar 2, 2015, at 12:28 PM, Kenneth D. Merry wrote: >=20 > On Mon, Mar 02, 2015 at 11:44:09 -0500, Dan Langille wrote: >>=20 >>> On Mar 2, 2015, at 11:31 AM, Kenneth D. Merry = wrote: >>>=20 >>> On Mon, Mar 02, 2015 at 11:09:57 -0500, Dan Langille wrote: >>>>=20 >>>>> On Mar 1, 2015, at 9:29 PM, Kenneth D. Merry = wrote: >>>>>=20 >>>>> On Sun, Mar 01, 2015 at 19:41:07 -0500, Dan Langille wrote: >>>>>>=20 >>>>>>> On Mar 1, 2015, at 7:31 PM, Kenneth D. Merry = wrote: >>>>>>>=20 >>>>>>> On Sun, Mar 01, 2015 at 19:15:05 -0500, Dan Langille wrote: >>>>>>>>=20 >>>>>>>>> On Feb 17, 2015, at 1:36 PM, Kenneth D. Merry = wrote: >>>>>>>>>=20 >>>>>>>>> On Sat, Feb 14, 2015 at 18:22:43 -0500, Dan Langille wrote: >>>>>>>>>>=20 >>>>>>>>>>> On Feb 13, 2015, at 7:32 PM, Kenneth D. Merry = wrote: >>>>>>>>>>>=20 >>>>>>>>>>>=20 >>>>>>>>>>> I have a fairly large set of changes to the sa(4) driver and = mt(1) driver >>>>>>>>>>> that I'm planning to commit in the near future. >>>>>>>>>>>=20 >>>>>>>>>>> A description of the changes is here and below in this = message. >>>>>>>>>>>=20 >>>>>>>>>>> If you have tape hardware and the inclination, I'd = appreciate testing and >>>>>>>>>>> feedback. >>>>>>>>>>=20 >>>>>>>>>> I have a DLT 8000 and an SDLT 220. >>>>>>>>>>=20 >>>>>>>>>> I don't have anything running current, but I have a spare = machine which I could use for testing. >>>>>>>>>>=20 >>>>>>>>>> Do you see any value is tests with that hardware? I'd be = testing it via Bacula. >>>>>>>>>>=20 >>>>>>>>>> disclosure: I'm the sysutils/bacula-* maintainer and a Bacula = committer. >>>>>>>>>>=20 >>>>>>>>>=20 >>>>>>>>> Actually, yes. Bacula is a bit tricky to configure, so your = trying it out >>>>>>>>> would be helpful if you have the time. >>>>>>>>>=20 >>>>>>>>> In looking at the manuals for both the SDLT 220 and the DLT = 8000, they both >>>>>>>>> claim to support long position information for the SCSI READ = POSITION >>>>>>>>> command. >>>>>>>>>=20 >>>>>>>>> You can see what I'm talking about by doing: >>>>>>>>>=20 >>>>>>>>> mt eod >>>>>>>>> mt status >>>>>>>>>=20 >>>>>>>>> On my DDS-4 tape drive, this shows: >>>>>>>>>=20 >>>>>>>>> # mt -f /dev/nsa3 status >>>>>>>>> Drive: sa3: Serial Number: = HJ00YWY >>>>>>>>> --------------------------------- >>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>> Current: 0x26:DDS-4 1024 bytes 97000 enabled = (DCLZ) >>>>>>>>> --------------------------------- >>>>>>>>> Current Driver State: at rest. >>>>>>>>> --------------------------------- >>>>>>>>> Partition: 0 Calc File Number: -1 Calc Record = Number: -1 >>>>>>>>> Residual: 0 Reported File Number: -1 Reported Record = Number: -1 >>>>>>>>> Flags: None >>>>>>>>>=20 >>>>>>>>> But on an LTO-5, which will give long position information, I = get: >>>>>>>>>=20 >>>>>>>>> [root@doc ~]# mt status >>>>>>>>> Drive: sa0: >>>>>>>>> --------------------------------- >>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>> Current: 0x58:LTO-5 variable 384607 enabled = (0x1) >>>>>>>>> --------------------------------- >>>>>>>>> Current Driver State: at rest. >>>>>>>>> --------------------------------- >>>>>>>>> Partition: 0 Calc File Number: 2 Calc Record = Number: -1 >>>>>>>>> Residual: 0 Reported File Number: 2 Reported Record = Number: 32373 >>>>>>>>> Flags: None >>>>>>>>>=20 >>>>>>>>> That, in combination with the changes I made to the position = information >>>>>>>>> code in the driver, mean that even the old MTIOCGET ioctl = should return an >>>>>>>>> accurate file number at end of data. e.g., on the LTO-5: >>>>>>>>>=20 >>>>>>>>> [root@doc ~]# mt ostatus >>>>>>>>> Mode Density Blocksize bpi = Compression >>>>>>>>> Current: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> ---------available modes--------- >>>>>>>>> 0: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> 1: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> 2: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> 3: 0x58:LTO-5 variable 384607 0x1 >>>>>>>>> --------------------------------- >>>>>>>>> Current Driver State: at rest. >>>>>>>>> --------------------------------- >>>>>>>>> File Number: 2 Record Number: -1 Residual Count -1 >>>>>>>>>=20 >>>>>>>>> So the thing to try, in addition to just making sure that = Bacula continues >>>>>>>>> to work properly, is to try setting this for the tape drive in >>>>>>>>> bacula-sd.conf: >>>>>>>>>=20 >>>>>>>>> Hardware End of Medium =3D yes >>>>>>>>>=20 >>>>>>>>> It looks like the Bacula tape program (btape) has a test mode, = and it would >>>>>>>>> be good to run through the tests on one of the tape drives and = see whether >>>>>>>>> they work, and whether the results are different before and = after the >>>>>>>>> changes. I'm not sure how to enable the test mode. >>>>>>>>=20 >>>>>>>> I have this in /usr/local/etc/bacula/bacula-sd.conf >>>>>>>>=20 >>>>>>>> Device { >>>>>>>> Name =3D DLT >>>>>>>> Description =3D "QUANTUM DLT7000 1624" >>>>>>>> Media Type =3D DLT >>>>>>>> Archive Device =3D /dev/nsa1 >>>>>>>>=20 >>>>>>>> Autochanger =3D YES >>>>>>>> Drive Index =3D 0 >>>>>>>>=20 >>>>>>>> Offline On Unmount =3D no >>>>>>>> Hardware End of Medium =3D yes >>>>>>>> BSF at EOM =3D yes >>>>>>>> Backward Space Record =3D no >>>>>>>> Fast Forward Space File =3D no >>>>>>>> TWO EOF =3D yes >>>>>>>> } >>>>>>>>=20 >>>>>>>> FYI, http://www.freebsddiary.org/digital-tl891.php (from 2006) = has a btape test on this same model. >>>>>>>>=20 >>>>>>>> Here's the test I ran tonight: >>>>>>>>=20 >>>>>>>> [root@cuppy:/usr/home/dan] # btape -c = /usr/local/etc/bacula/bacula-sd.conf /dev/nsa1 = =20 >>>>>>>> Tape block granularity is 1024 bytes. >>>>>>>> btape: butil.c:287-0 Using device: "/dev/nsa1" for writing. >>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>> *test >>>>>>>>=20 >>>>>>>> =3D=3D=3D Write, rewind, and re-read test =3D=3D=3D >>>>>>>>=20 >>>>>>>> I'm going to write 10000 records and an EOF >>>>>>>> then write 10000 records and an EOF, then rewind, >>>>>>>> and re-read the data to verify that it is correct. >>>>>>>>=20 >>>>>>>> This is an *essential* feature ... >>>>>>>>=20 >>>>>>>> btape: btape.c:1152-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1168-0 Wrote 10000 blocks of 64412 bytes. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1210-0 Rewind OK. >>>>>>>> 10000 blocks re-read correctly. >>>>>>>> Got EOF on tape. >>>>>>>> 10000 blocks re-read correctly. >>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>=20 >>>>>>>> btape: btape.c:1277-0 Block position test >>>>>>>> btape: btape.c:1289-0 Rewind OK. >>>>>>>> Reposition to file:block 0:4 >>>>>>>> Block 5 re-read correctly. >>>>>>>> Reposition to file:block 0:200 >>>>>>>> Block 201 re-read correctly. >>>>>>>> Reposition to file:block 0:9999 >>>>>>>> Block 10000 re-read correctly. >>>>>>>> Reposition to file:block 1:0 >>>>>>>> Block 10001 re-read correctly. >>>>>>>> Reposition to file:block 1:600 >>>>>>>> Block 10601 re-read correctly. >>>>>>>> Reposition to file:block 1:9999 >>>>>>>> Block 20000 re-read correctly. >>>>>>>> =3D=3D=3D Test Succeeded. End Write, rewind, and re-read test = =3D=3D=3D >>>>>>>>=20 >>>>>>>>=20 >>>>>>>>=20 >>>>>>>> =3D=3D=3D Append files test =3D=3D=3D >>>>>>>>=20 >>>>>>>> This test is essential to Bacula. >>>>>>>>=20 >>>>>>>> I'm going to write one record in file 0, >>>>>>>> two records in file 1, >>>>>>>> and three records in file 2 >>>>>>>>=20 >>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:1907-0 Wrote one record of 64412 bytes. >>>>>>>> btape: btape.c:1909-0 Wrote block to device. >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:604-0 Wrote 1 EOF to "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:469-0 open device "DLT" (/dev/nsa1): OK >>>>>>>> btape: btape.c:574-0 Rewound "DLT" (/dev/nsa1) >>>>>>>> btape: btape.c:1420-0 Now moving to end of medium. >>>>>>>=20 >>>>>>> This is the critical piece. The test moves the tape to the end = of the >>>>>>> medium. With hardware position information, you can tell what = filemark >>>>>>> you're on. Without it, you can't. >>>>>>>=20 >>>>>>>> btape: btape.c:622-0 tape_dev.c:345 ioctl MTIOCGET error on = "DLT" (/dev/nsa1). ERR=3DNo error: 0. >>>>>>>> We should be in file 3. I am at file 0. This is NOT correct!!!! >>>>>>>>=20 >>>>>>>> Append test failed. Attempting again. >>>>>>>> Setting "Hardware End of Medium =3D no >>>>>>>> and "Fast Forward Space File =3D no >>>>>>>> and retrying append test. >>>>>>>=20 >>>>>>> This is not surprsing, given that the drive doesn't support long = read >>>>>>> position data. (It's a SCSI-2 device.) So that means that = Bacula will >>>>>>> need to do it manually. >>>>>>=20 >>>>>> Yes, I have nothing newer than SCSI-2. Even my SDLT is SCSI-2 = but that >>>>>> tape library is hooked up to a different computer and was doing = backups today. >>>>>=20 >>>>> So, here is one thing that we can try to see whether these drives = support >>>>> long position information, even though they only claim to be = SCSI-2. If >>>>> they do, we can potentially add a quirk (or autodetection) to = enable it. >>>>> The code currently doesn't bother asking drives that claim to be = SCSI-2 >>>>> for long position information. (Because that feature was added in = the >>>>> SSC spec, which came after SCSI-2.) >>>>>=20 >>>>> Issue a READ POSITION with the short form specified: >>>>>=20 >>>>> camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>=20 >>>>> Issue a READ POSITION with the vendor-specific block numbers: >>>>>=20 >>>>> camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i 20 - |hd >>>>>=20 >>>>> Issue a READ POSITION with the long form data: >>>>>=20 >>>>> camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i 32 - |hd >>>>>=20 >>>>> If it supports the last one, then I can put a quirk (or = autodetection) in >>>>> the driver and Bacula will get the hardware filemarks. You should = try this >>>>> on your SDLT as well. It may well support it. >>>>=20 >>>> Sadly, no: >>>>=20 >>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 0 0 0 0" -i = 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 1 0 0 0 0 0 0 0 0" -i = 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@cuppy:~] # camcontrol cmd sa1 -v -c "34 6 0 0 0 0 0 0 0 0" -i = 32 - |hd >>>> camcontrol: error sending command >>>> (pass2:ahc0:0:2:0): READ POSITION. CDB: 34 06 00 00 00 00 00 00 00 = 00=20 >>>> (pass2:ahc0:0:2:0): CAM status: SCSI Status Error >>>> (pass2:ahc0:0:2:0): SCSI status: Check Condition >>>> (pass2:ahc0:0:2:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid = field in CDB) >>>> (pass2:ahc0:0:2:0): Command byte 1 bit 2 is invalid >>>> [root@cuppy:~] #=20 >>>=20 >>> Okay. Not too surprising I suppose. Does this mean much to you? Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Sending SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: (ahc0:A:2:0): Received SDTR period 19, = offset f Mar 2 19:42:59 cuppy kernel: Filtered to period 19, offset f I'm having trouble with labeling barcodes from Bacula and the above is = seen in /var/log/messages >>>=20 >>>> The SDLT server is on 9.3 though: >>>>=20 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa1 -v -c "34 0 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>> camcontrol: cam_lookup_pass: CAMGETPASSTHRU ioctl failed >>>> cam_lookup_pass: No such file or directory >>>> cam_lookup_pass: either the pass driver isn't in your kernel >>>> cam_lookup_pass: or sa1 doesn't exist >>>> [root@knew:/usr/home/dan] # uname -a >>>> FreeBSD knew.unixathome.org 9.3-RELEASE-p10 FreeBSD 9.3-RELEASE-p10 = #0: Tue Feb 24 21:28:03 UTC 2015 = root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 >>>> [root@knew:/usr/home/dan] #=20 >>>>=20 >>>>=20 >>>> It took me a while to figure that out... there is no sa1 on *this* = system. >>>>=20 >>>> But, my SDLT: >>>>=20 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 0 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 1 0 0 0 0 = 0 0 0 0" -i 20 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 |....| >>>> 00000014 >>>> [root@knew:/usr/home/dan] # camcontrol cmd sa0 -v -c "34 6 0 0 0 0 = 0 0 0 0" -i 32 - |hd >>>> 00000000 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| >>>> 00000020 >>>> [root@knew:/usr/home/dan] #=20 >>>=20 >>> Just to confirm, can you send the output of: >>>=20 >>> camcontrol inquiry sa0 -v >>>=20 >>> I want to make certain it reports that it is SCSI-2. If so, I'll = change >>> the check in the driver to try asking for long position information = on >>> SCSI-2 devices. If it fails, it'll fall back to the regular method. >>=20 >> [dan@knew:~] $ sudo camcontrol inquiry sa0 -v >> pass10: Removable Sequential Access SCSI-2 = device=20 >> pass10: Serial Number CXB46H0716 =20 >> pass10: 80.000MB/s transfers (40.000MHz, offset 31, 16bit) >> [dan@knew:~] $=20 >=20 > Okay. Doing the check doesn't cause any problems on my collection of = old > tape drives, so I'll go ahead and enable checking on SCSI-2 devices. >=20 > The primary thing is just making sure it doesn't cause tape drive to = choke > if we request long position information. It doesn't appear to be an > issue. If we do run into one, we can put in a quirk. >=20 > Ken > --=20 > Kenneth Merry > ken@FreeBSD.ORG =E2=80=94=20 Dan Langille http://langille.org/