Date: Mon, 23 Jun 2003 11:28:43 -0600 From: Bob Bawn <bbawn@allocity.com> To: freebsd-scsi@freebsd.org Subject: problem with large aio_write(2)s to raw device through Compaq ciss driver Message-ID: <3EF738CB.3090901@allocity.com>
next in thread | raw e-mail | index | archive | help
Hello,
(I hope this is the right forum for this issue - I tried
freebsd-questions a couple weeks ago and got no response.)
I am running FreeBSD 4.7 on a Compaq DL 380 with a Compaq
Smart Array 5i.
My application accesses a raw device (e.g. /dev/da0s1g) using
aio_write(2).
aio_writes of buffers larger than 224 (512-byte) blocks but
smaller than 257 blocks fail with EIO. I get the following
messages when this happens:
bus_dmamap_load: Too many segs! buf_len = 0x3000
ciss0: invalid command, offense size 0 at 52, value 0x0
These writes succeed on various other hardware configurations
(Dell RAID, SCSI disk, IDE disk, etc.) so I suspect the ciss driver.
Synchronous (write(2)) writes in this size range to the raw device
succeed.
aio_writes to normal files succeed.
Glancing through the ciss source, I noticed that 224 * 512 = 28 * 4096
where 28 is CISS_COMMAND_SG_LENGTH (the max number of scatter/gather
elements per command??). So maybe the write fails if the s/g vector
doesn't fit in a single command? (I am a non-expert in this area, so
this is speculative...)
I don't understand why writes larger than 256 blocks succeed.
The following patch seems to fix the problem:
*** /usr/src/sys/dev/ciss/cissvar.h.orig Mon Jun 16 14:16:39 2003
--- /usr/src/sys/dev/ciss/cissvar.h Mon Jun 16 14:21:40 2003
***************
*** 140,146 ****
* too small.
*/
! #define CISS_COMMAND_ALLOC_SIZE 512 /* XXX tune to
get sensible s/g list length */
#define CISS_COMMAND_SG_LENGTH ((CISS_COMMAND_ALLOC_SIZE -
sizeof(struct ciss_command)) \
/ sizeof(struct ciss_sg_entry))
--- 140,153 ----
* too small.
*/
! /*
! * 6/16/03 bbawn - aio_write(2)s between 225 and 256 blocks (inclusive)
! * fail with EIO with CISS_COMMAND_ALLOC_SIZE of 512. Fix (or actually
kludge
! * around this) by having room for enough scatter/gather entries to
! * exceed 256 blocks) (the max size for a SCSI WRITE(6) command??).
! * #define CISS_COMMAND_ALLOC_SIZE 512
! */
! #define CISS_COMMAND_ALLOC_SIZE 1024 /* XXX tune to
get sensible s/g list length */
#define CISS_COMMAND_SG_LENGTH ((CISS_COMMAND_ALLOC_SIZE -
sizeof(struct ciss_command)) \
/ sizeof(struct ciss_sg_entry))
Any clues on what's going on here? (and more importantly, if my "fix" is
adequate?) If time permits, I hope to investigate in the debugger but
any information would be appreciated.
I have a small program that illustrates the problem - let me know
if you want it.
It seems possible that I have something mis-configured. Here are the
boot messages from ciss:
Jun 6 10:00:16 queso /kernel: pci0: <PCI bus> on pcib0
Jun 6 10:00:16 queso /kernel: ciss0: <Compaq Smart Array 5i> port
0x2000-0x20ff mem 0xf5ef0000-0xf5ef3fff,0xf7ec0000-0xf7efffff irq 3 at
device 1.0 on pci0
Jun 6 10:00:16 queso /kernel: ciss0: using 256 of 1024 available commands
Jun 6 10:00:16 queso /kernel: ciss0: 3 logical drives configured
Jun 6 10:00:16 queso /kernel: ciss0: firmware 1.92
Jun 6 10:00:16 queso /kernel: ciss0: 2 SCSI channels
Jun 6 10:00:16 queso /kernel: ciss0: signature 'CISS'
Jun 6 10:00:16 queso /kernel: ciss0: valence 1
Jun 6 10:00:16 queso /kernel: ciss0: supported I/O methods
0xe<simple,performant,MEMQ>
Jun 6 10:00:16 queso /kernel: ciss0: active I/O method 0x3<simple>
Jun 6 10:00:16 queso /kernel: ciss0: 4G page base 0x00000000
Jun 6 10:00:16 queso /kernel: ciss0: interrupt coalesce delay 1000us
Jun 6 10:00:16 queso /kernel: ciss0: interrupt coalesce count 16
Jun 6 10:00:16 queso /kernel: ciss0: max outstanding commands 1024
Jun 6 10:00:16 queso /kernel: ciss0: bus types 0x2<ultra3>
Jun 6 10:00:16 queso /kernel: ciss0: server name ''
Jun 6 10:00:16 queso /kernel: ciss0: heartbeat 0x10000033
Jun 6 10:00:16 queso /kernel: ciss0: 3 logical drives
Jun 6 10:00:16 queso /kernel: ciss0: logical drive 0: RAID 5, 92160MB
online
Jun 6 10:00:16 queso /kernel: ciss0: logical drive 1: RAID 5, 92160MB
online
Jun 6 10:00:16 queso /kernel: ciss0: logical drive 2: RAID 5, 92160MB
online
Thanks,
Bob Bawn
bbawn@allocity.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3EF738CB.3090901>