Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 6 Apr 2015 16:07:44 -0600
From:      "Kenneth D. Merry" <ken@FreeBSD.ORG>
To:        Fabian Keil <freebsd-listen@fabiankeil.de>
Cc:        freebsd-current@freebsd.org
Subject:   Re: async pass(4) patches available
Message-ID:  <20150406220744.GA92212@mithlond.kdm.org>
In-Reply-To: <2147223b.4518a234@fabiankeil.de>
References:  <20150330222358.GA46342@mithlond.kdm.org> <2147223b.4518a234@fabiankeil.de>

next in thread | previous in thread | raw e-mail | index | archive | help

--GvXjxJ+pjyke8COw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Mon, Apr 06, 2015 at 15:39:56 +0200, Fabian Keil wrote:
> "Kenneth D. Merry" <ken@FreeBSD.ORG> wrote:
> 
> > I have put patches to add an asynchronous interface to the pass(4) driver
> > and add a new camdd(8) utility here:
> > 
> > FreeBSD/head as of SVN revision 280857:
> > 
> > http://people.freebsd.org/~ken/async_pass.head.20150330.1.txt
> [...]
> > Comments and testing are welcome!  As I said, camdd(8) in particular is a
> > work in progress.  It could use some cleanup and there are some more
> > useful features that could be added there.
> 
> I've been using the patch for a couple of days on an amd64 system
> based on 11.0-CURRENT r280952 and didn't notice any obvious
> regressions using the system as usual.
> 
> Scrubbing a pool once revealed checksum errors which I haven't
> seen before:
> 
> [fk@kendra ~]$ zpool status -v dpool
>   pool: dpool
>  state: ONLINE
> status: One or more devices has experienced an unrecoverable error.  An
>         attempt was made to correct the error.  Applications are unaffected.
> action: Determine if the device needs to be replaced, and clear the errors
>         using 'zpool clear' or replace the device with 'zpool replace'.
>    see: http://illumos.org/msg/ZFS-8000-9P
>   scan: scrub repaired 0 in 1h52m with 0 errors on Thu Apr  2 13:01:44 2015
> config:
> 
>         NAME                  STATE     READ WRITE CKSUM
>         dpool                 ONLINE       0     0     0
>           gpt/dpool-ada0.eli  ONLINE       0     0     6
> 
> errors: No known data errors
> 
> Apr  2 12:31:34 kendra kernel: (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 30 17 61 55 40 31 00 00 00 00 00
> Apr  2 12:31:34 kendra kernel: (ada0:ahcich0:0:0:0): CAM status: ATA Status Error
> Apr  2 12:31:34 kendra kernel: (ada0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 40 (UNC )
> Apr  2 12:31:34 kendra kernel: (ada0:ahcich0:0:0:0): RES: 51 40 3e 61 55 40 31 00 00 00 00
> Apr  2 12:31:34 kendra kernel: (ada0:ahcich0:0:0:0): Error 5, Retries exhausted
> Apr  2 12:31:34 kendra kernel: GEOM_ELI: g_eli_read_done() failed gpt/dpool-ada0.eli[READ(offset=414970949120, length=24576)]
> 
> However the issue doesn't seem to be (easily) reproducible
> and could be unrelated.

It is unlikely that this is related to the pass(4) driver patches.  Possible,
but highly unlikely.  camdd(8) doesn't support ATA passthrough yet, so the
only way to access it with camdd is with the file I/O method.

> I also tried to test camdd, but didn't get it to work.
> Some failed attempts:
> 
> [fk@kendra ~]$ sudo camdd -i pass=da0,bs=65536 -o file=blafsel.img
> (pass2:umass-sim0:0:0:0): READ(6). CDB: 08 00 00 00 80 00 
> (pass2:umass-sim0:0:0:0): CAM status: CCB request completed with an error
> 13 bytes read from pass2
> 13 bytes written to blafsel.img
> 20.3203 seconds elapsed
> 0.00 MB/sec
> [fk@kendra ~]$ sudo hd blafsel.img 
> 00000000  55 53 42 53 d9 02 00 00  00 00 01 00 01           |USBS.........|
> 0000000d
> [fk@kendra ~]$ sudo dd if=/dev/da0 bs=1k count=1  | hd | head -n 1
> 1+0 records in
> 1+0 records out
> 1024 bytes transferred in 0.000603 secs (1697756 bytes/sec)
> 00000000  fc 31 c0 8e c0 8e d8 8e  d0 bc 00 0e be 1a 7c bf  |.1............|.|

One possibility is that the device doesn't support 6-byte read/write
requests.  The da(4) driver has quirk entries and code to figure that out
and default to 10-byte read/write requests, but camdd(8) doesn't have
anything like that yet.

I've attached patches to camdd that allow you to specify a minimum command
size.  So, apply the patches, rebuild camdd, and try this:

# sudo camdd -i pass=da0,bs=65536,mcs=10 -o file=blafsel.img

We'll see if that helps.  I'm not sure why you were even able to get 13
bytes back.  That is very strange.

> Trying the block size suggested in the manual result in:
> 
> [fk@kendra ~]$ sudo camdd -i pass=da0,bs=1M -o file=blafsel.img
> camdd: camdd_pass_run: error sending CAMIOQUEUE ioctl to pass2: Invalid argument
> camdd: camdd_pass_run: CCB address is 0x80250e420: Invalid argument
> 0 bytes read from pass2
> 0 bytes written to blafsel.img
> 0.0007 seconds elapsed
> 0.00 MB/sec
> 
> Apr  5 19:08:20 kendra kernel: (pass2:umass-sim0:0:0:0): passmemsetup: data length 1048576 > max allowed 65536 bytes
> 

Yes.  By default, if you don't specify a blocksize, camdd(8) should limit
the I/O size to the controller's maximum or 128K, whichever is smaller.  If
you specify an I/O size, it will try to use that.

Thanks for testing the code, I really appreciate it!

Let me know how the patch works!

Ken
-- 
Kenneth Merry
ken@FreeBSD.ORG

--GvXjxJ+pjyke8COw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="camdd_mcs.20150406.1.txt"

==== //depot/users/kenm/FreeBSD-test2/usr.sbin/camdd/camdd.8#1 - /usr/home/kenm/perforce4/kenm/FreeBSD-test2/usr.sbin/camdd/camdd.8 ====
*** /tmp/tmp.54366.13	Mon Apr  6 21:56:38 2015
--- /usr/home/kenm/perforce4/kenm/FreeBSD-test2/usr.sbin/camdd/camdd.8	Mon Apr  6 21:23:29 2015
***************
*** 31,37 ****
  .\" 
  .\" $FreeBSD$
  .\"
! .Dd March 13, 2015
  .Dt CAMDD 8
  .Os
  .Sh NAME
--- 31,37 ----
  .\" 
  .\" $FreeBSD$
  .\"
! .Dd April 6, 2015
  .Dt CAMDD 8
  .Os
  .Sh NAME
***************
*** 39,45 ****
  .Nd CAM data transfer utility
  .Sh SYNOPSIS
  .Nm
! .Aq Fl i|o Ar pass=pass_dev|file=filename,bs=blocksize,offset=N,depth=N
  .Op Fl C Ar retry_count
  .Op Fl E
  .Op Fl m Ar max_io
--- 39,45 ----
  .Nd CAM data transfer utility
  .Sh SYNOPSIS
  .Nm
! .Aq Fl i|o Ar pass=pass_dev|file=filename,bs=blocksize,offset=N,depth=N,mcs=N
  .Op Fl C Ar retry_count
  .Op Fl E
  .Op Fl m Ar max_io
***************
*** 164,169 ****
--- 164,189 ----
  possible to maintain for especially fast devices.
  For writes, maintaining the queue depth also depends on a sufficiently
  fast reading device.
+ .It mcs=N
+ Specify the minimum command size to use for
+ .Xr pass 4
+ devices.
+ Some devices do not support 6 byte
+ .Tn SCSI
+ commands.
+ The
+ .Xr da 4
+ device handles this restriction automatically, but the
+ .Xr pass 4
+ device allows the user to specify the
+ .Tn SCSI
+ command used.
+ If a device does not accept 6 byte
+ .Tn SCSI
+ READ/WRITE commands (which is the default at lower LBAs), it will generally
+ accept 10 byte
+ .Tn SCSI
+ commands instead.
  .El
  .It Fl C Ar count
  Specify the retry count for commands sent via the asynchronous
==== //depot/users/kenm/FreeBSD-test2/usr.sbin/camdd/camdd.c#11 - /usr/home/kenm/perforce4/kenm/FreeBSD-test2/usr.sbin/camdd/camdd.c ====
*** /tmp/tmp.54366.104	Mon Apr  6 21:56:38 2015
--- /usr/home/kenm/perforce4/kenm/FreeBSD-test2/usr.sbin/camdd/camdd.c	Mon Apr  6 21:30:07 2015
***************
*** 110,115 ****
--- 110,116 ----
  	uint64_t	blocksize;
  	uint64_t	queue_depth;
  	uint64_t	offset;
+ 	int		min_cmd_size;
  	int		write_dev;
  };
  
***************
*** 306,311 ****
--- 307,313 ----
  	uint32_t		 sector_size;
  	uint64_t		 max_sector;
  	uint64_t		 sector_io_limit;
+ 	int			 min_cmd_size;
  	int			 write_dev;
  	int			 retry_count;
  	int			 io_timeout;
***************
*** 1448,1453 ****
--- 1450,1457 ----
  #endif
  	}
  
+ 	dev->min_cmd_size = io_opts->min_cmd_size;
+ 
  	dev->run = camdd_pass_run;
  	dev->fetch = camdd_pass_fetch;
  
***************
*** 2317,2323 ****
  			/*readop*/ (dev->write_dev == 0) ? SCSI_RW_READ :
  				   SCSI_RW_WRITE,
  			/*byte2*/ 0,
! 			/*minimum_cmd_size*/ 0,
  			/*lba*/ buf->lba,
  			/*block_count*/ num_blocks,
  			/*data_ptr*/ (data->sg_count != 0) ?
--- 2321,2327 ----
  			/*readop*/ (dev->write_dev == 0) ? SCSI_RW_READ :
  				   SCSI_RW_WRITE,
  			/*byte2*/ 0,
! 			/*minimum_cmd_size*/ dev->min_cmd_size,
  			/*lba*/ buf->lba,
  			/*block_count*/ num_blocks,
  			/*data_ptr*/ (data->sg_count != 0) ?
***************
*** 3048,3053 ****
--- 3052,3058 ----
  "offset=len    Specify starting offset in bytes or using K, M, G suffix\n"
  "              NOTE: offset cannot be specified on tapes, pipes, stdin/out\n"
  "depth=N       Specify a numeric queue depth.  This only applies to pass(4)\n"
+ "mcs=N         Specify a minimum cmd size for pass(4) read/write commands\n"
  "Optional arguments\n"
  "-C retry_cnt  Specify a retry count for pass(4) devices\n"
  "-E            Enable CAM error recovery for pass(4) devices\n"
***************
*** 3128,3134 ****
  
  			io_opts->queue_depth = strtoull(value, &endptr, 0);
  			if (*endptr != '\0') {
! 				warn("invalid queue depth %s", value);
  				retval = 1;
  				goto bailout;
  			}
--- 3133,3150 ----
  
  			io_opts->queue_depth = strtoull(value, &endptr, 0);
  			if (*endptr != '\0') {
! 				warnx("invalid queue depth %s", value);
! 				retval = 1;
! 				goto bailout;
! 			}
! 		} else if (strncasecmp(name, "mcs", 3) == 0) {
! 			char *endptr;
! 
! 			io_opts->min_cmd_size = strtol(value, &endptr, 0);
! 			if ((*endptr != '\0')
! 			 || ((io_opts->min_cmd_size > 16)
! 			  || (io_opts->min_cmd_size < 0))) {
! 				warnx("invalid minimum cmd size %s", value);
  				retval = 1;
  				goto bailout;
  			}

--GvXjxJ+pjyke8COw--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150406220744.GA92212>