Date: Thu, 03 Nov 2005 10:27:48 -0800 From: Don Murray <donm@ptgrey.com> To: aic7xxx@freebsd.org Subject: re: Non-adaptec card recommendation Message-ID: <436A56A4.1050403@ptgrey.com> In-Reply-To: <430275CB.4040706@geeksrus.ca> References: <430275CB.4040706@geeksrus.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi all - I just wanted to follow up on this with the current status
although the jury is still out on the final result.
I did increase the TQD depth as Todd recommended a few weeks ago. Since
doing that the situation has seemed better in that I have not "lost" the
disk at all. However, a few days users complained to me that the disk
access would hang for several seconds when they tried to save their
work. I tried to disable DV as well, following the comments by Todd,
but I don't think I have my options string right because DV was still
happening.
Anyway, I went ahead and got a LSI Logic (LSI22320RB) scsi adapter and
installed it last night. This morning I tried the performance test
software provided by www.passmark.com again. (This is testing disk
access from a windows machine over samba.) Previously I would get
results of about 8 MB/s, except for their sequential write test that
just bogged down horribly and gave a result of around 0.3 MB/s (even
with the modified TQD). With the LSI, I re-ran the test and got around
11 MB/s across all the tests (read, write, random rw). This indicated
that maybe it was being limited by the 100Mb ethernet access. I re-ran
the test on a windows server that is connected to the file server via a
gigabit ethernet connection and the rates went to 100MB/s read and
60MB/s write! So, speed wise, just switching to the LSI board seems to
have done a great job. The jury is still out as far as stability goes.
But I've found that these errors:
Nov 1 12:14:45 windsor smbd[28163]: [2005/11/01 12:14:45, 0]
lib/util_sock.c:read_socket_data(384)
Nov 1 12:14:45 windsor smbd[28163]: read_socket_data: recv failure
for 4. Error = Connection reset by peer
have disappeared so far with the new card so I have high hopes. Note -
I'm using a different disk array than Todd so we are probably playing
with different firmware. I'm using the VTrak 12110 which does not have
any firmware updates.
BTW, I don't seem to be properly subscribed to this list as I don't seem
to be receiving the regular traffic, but luckily I go to check the
archives every couple of weeks.
Also, I want to thank Todd and Ken for the information! Todd's
description of what he has done to improve the problem really helped me
understand more about the issues involved in improving disk performance.
Thanks again,
Don
Ken Seefried wrote:
>>/
>/>/ Don wrote:
>/>/
>/>/ >
>/>/ > I guess this is a funny thing to request on this list, but does anyone
>/>/ > have any recommendations for a SCSI card manufacturer other than
>/>/ > Adaptec that would work well with Linux?
>/>/
>/>/ I've found LSI Logic chipset based SCSI cards to be well supported, at
>/>/ least for the ones I've used. YMMV.
>/
>Unfortunately if you check the BIOS updates from Promise (I am assuming Don
>is still trying to get that to work) the LSI boards also can drive the
>Promise to fault. At least one of the BIOS upgrades for the RM8000 series
>indicated they had made changes to stabilize the array when controlled by
>adaptec OR LSI under Linux/BSD. I have come to believe that Linux & BSD can
>drive the IO subsystems faster than the same hardware (whole box) than
>windows or even HP-UX. Because I believe that to make the promise drives
>stable you will need to change your OS, not your hardware, I am including
>again my latest list of how I stabilize the drives fairly well under Linux.
>
>
>
>So far, I have been able to extend the Promise drive's MTBLockUp to several
>weeks (from a starting position of 8 hours of real use between locks) by:
>
>1) Update the Firmware in the Promise device to their latest, as the do tend
>to make the devices more stable so the firmware writers must be paying a
>_little_ attention to the customers with problems.
>
>2) turn off Domain Validation(DV) in /etc/modules.conf, even though DV is
>required for a U160 device and faster, promise representatives[1] don't seem
>to think it is necessary. And because they don't handle it, sending DV
>messages destabilizes the drive firmware.
>i.e., put the following line in your /etc/modules.conf
>options aic7xxx aic7xxx=verbose.global_tag_depth:16.dv:{0}
>
>3) rebuild the initrd(at least on redhat/fedora), so that the values in the
>modules.conf are used on boot:
>/sbin/new-kernel-pkg --mkinitrd --depmod --install <kernel name>
>
>4) do the work of Domain Validation by hand, go into the bios setup of the
>card and slow the scsi bus down from the drives maximum throughput. Mine had
>to be slowed down from 160 to 53, and I would have made it 40 or 20 but the
>next lower the Promise would talk at was 16.
>
>5) Play with the size of the tag queue. I found that the average write rate
>went from ~14MB/s to ~27.1MB/s when I changed the queue size from 224 to 16,
>also 1 tag got me ~27.1MB/s, 8 tags got me ~25.7MB/s, 32 tags got me
>~25.8MB/s and 64 got me ~24.6MB/s, so I settled on 16. It also seems some
>what more stable after shrinking the queue.
>
>
>
>
>[1] at least the ones I talked to on the phone.
>--
>Todd Denniston
>Crane Division, Naval Surface Warfare Center (NSWC Crane)
>Harnessing the Power of Technology for the Warfighter
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?436A56A4.1050403>