From owner-freebsd-stable@FreeBSD.ORG Thu Aug 18 22:20:18 2005 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3DBDC16A41F for ; Thu, 18 Aug 2005 22:20:18 +0000 (GMT) (envelope-from robert.hutterer@univie.ac.at) Received: from mx03.kabsi.at (mx03.kabsi.at [195.202.128.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E3D743D45 for ; Thu, 18 Aug 2005 22:20:17 +0000 (GMT) (envelope-from robert.hutterer@univie.ac.at) Received: from virtual (h062040150223.kob.cm.kabsi.at [62.40.150.223]) by mx03.kabsi.at (8.13.3/8.13.3) with SMTP id j7IMKDjl012995; Fri, 19 Aug 2005 00:20:14 +0200 Message-ID: <012f01c5a443$15a97b80$0901a8c0@virtual> From: "Hutterer Robert" To: "Justin T. Gibbs" , References: <00e901c5a1cd$94e1c9c0$0901a8c0@virtual> <1F21DAB5B24D156A1C27045D@aslan.scsiguy.com> Date: Fri, 19 Aug 2005 00:20:41 +0200 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2527 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 Cc: Subject: Re: DELL SC430 & ahd0: X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2005 22:20:18 -0000 Thank you very much for the reaction (about a dozen user reported similar problems the last month -but there seems no answer/solution) >>From what I can tell from the full card dump state, the 39320 attempted > to send 77 transactions to your drive during a single connection. This > connection hung, and the timeout occurred. Since the drive controlls > the connection, it can cut the initiator off at any time if too many > commands are sent. That seems plausilbe also for a non-expert > So, this looks like a drive firmware bug. You > should contact Dell to find out if newer firmware is available for your > drive Contacted Dell but they have no idea to fix this - freebsd is not supported by dell -directed me to adaptec. So I used the latest bios for the 39320 adapter from adaptec. ===================================================================== = Adaptec Ultra320 Family SCSI Controller = = PnP/BBS BIOS Version 4.30.0, P/N 2038403-00 Rev. AA = ===================================================================== Soon after a reboot I got similar but slightly different messages (see below - hope you understand it). I will see if I will get it more frequently > drive. In the mean time, you can try lowering the tag depth allowed > for your device. If the failure occurs sometime after rc processing, > you can make a call early in the transition to multi-user like so: > > camcontrol tags da0 -N 64 # or some lower number Unfortunately I am not that expert to understand what to do with this "call": to put it on the command line? To ma a startup command ? > If that won't work for you, you can enter a quirk into sys/cam/cam_xpt.c > or just modify the last quirk entry (the default) to have a lower tag > depth (it is currently 255). Also this hint I do not understand (I found (/usr/src/sys/cam/cam_xpr.c file) maybe you can give me an idea or direct me to some instruction pages how to enter a quirl or modify the last quirk entry > If you do get new firmware for your drive and that resolves this issue, > please follow up to the list (and cc me if you don't mind) so that > others that encounter the same issue can benefit from your experience. I am not optimistic for that. Dell used the bios from adaptec. Adaptec said this is the first time they heard from that and it seems that no other os is affected. If nothing helps I will seriously think about changing to a SATA disk. (But it is strange I have 39320 on a dell SC1420 and there is no problem) Thanks Robert xxxxxxxxxxxxxxxx MESSAGES BEGIN xxxxxxxxxxxxxxxx ahd0: Recovery Initiated - Card was not paused >>>>>>>>>>>>>>>>>> Dump Card State Begins <<<<<<<<<<<<<<<<< ahd0: Dumping Card State at program address 0x38 Mode 0x22 INTSTAT[0x0] SELOID[0x0] SELID[0x0] HS_MAILBOX[0x0] INTCTL[0x80]:(SWTMINTMASK) SEQINTSTAT[0x0] SAVED_MODE[0x11] DFFSTAT[0x33]:(CURRFIFO_NONE|FIFO0FREE|FIFO1FREE) SCSISIGI[0x27]:(P_DATAOUT_DT|ACKI|REQI|BSYI) SCSIPHASE[0x0] SCSIBUS[0xc7] LASTPHASE[0x1]:(P_DATAOUT|P_BUSFREE) SCSISEQ0[0x40]:(ENSELO) SCSISEQ1[0x12]:(ENAUTOATNP|ENRSELI) SEQCTL0[0x0] SEQINTCTL[0x0] SEQ_FLAGS[0x0] SEQ_FLAGS2[0x0] QFREEZE_COUNT[0x2] KERNEL_QFREEZE_COUNT[0x2] MK_MESSAGE_SCB[0xff00] MK_MESSAGE_SCSIID[0xff] SSTAT0[0x10]:(SELINGO) SSTAT1[0x8]:(BUSFREE) SSTAT2[0x0] SSTAT3[0x0] PERRDIAG[0x0] SIMODE1[0xa4]:(ENSCSIPERR|ENSCSIRST|ENSELTIMO) LQISTAT0[0x0] LQISTAT1[0x0] LQISTAT2[0x80]:(PACKETIZED) LQOSTAT0[0x0] LQOSTAT1[0x0] LQOSTAT2[0xc0] SCB Count = 256 CMDS_PENDING = 82 LASTSCB 0x17 CURRSCB 0x1d NEXTSCB 0xff40 qinstart = 3008 qinfifonext = 3008 QINFIFO: WAITING_TID_QUEUES: 0 ( 0x4a 0xa 0x75 0x6 0x76 0x49 0x9 0x3 0x69 0x24 0x6a 0x5f 0x40 0x51 0x52 0x53 0x54 0x55 0x56 0x78 0x79 0x71 0x2 0x45 0x46 0x35 0x0 0xc 0x14 0x5b 0x5c 0x20 0x21 0x27 0x34 0x3c 0x2e 0x22 0x23 0x68 0x3b 0x59 0x5a 0x50 0x43 0x41 0xf 0x6b 0x6c 0x6d 0x1c 0x18 0x44 0x42 0x7a 0x62 0x63 0x8 0x1e 0x67 0x60 0x61 0x28 0x6f 0x47 0x48 0x36 0x37 0x13 0x10 0x11 0x2a 0x2f 0x66 0x5e 0x5 0x17 0x1d 0x5d 0x4d 0x4e 0x4f ) Pending list: 79 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 78 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 77 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 93 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 29 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 23 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 5 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 94 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 102 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 47 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 42 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 17 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 16 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 19 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 55 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 54 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 72 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 71 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 111 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 40 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 97 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 96 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 103 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 30 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 8 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 99 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 98 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 122 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 66 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 68 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 24 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 28 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 109 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 108 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 107 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 15 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 65 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 67 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 80 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 90 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 89 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 59 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 104 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 35 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 34 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 46 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 60 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 52 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 39 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 33 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 32 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 92 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 91 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 20 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 12 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 0 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 53 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 70 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 69 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 2 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 113 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 121 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 120 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 86 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 85 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 84 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 83 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 82 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 81 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 64 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 95 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 106 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 36 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 105 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 3 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 9 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 73 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 118 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 6 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 117 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 10 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] 74 FIFO_USE[0x0] SCB_CONTROL[0x60]:(TAG_ENB|DISCENB) SCB_SCSIID[0x7] Total 82 Kernel Free SCB list: 49 13 119 11 21 38 87 88 43 50 51 112 101 25 37 22 27 7 4 110 1 31 125 126 127 14 18 48 61 62 63 56 57 58 44 45 115 116 123 124 41 26 75 76 100 114 143 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 Sequencer Complete DMA-inprog list: Sequencer Complete list: Sequencer DMA-Up and Complete list: Sequencer On QFreeze and Complete list: ahd0: FIFO0 Free, LONGJMP == 0x8276, SCB 0x4a SEQIMODE[0x3f]:(ENCFG4TCMD|ENCFG4ICMD|ENCFG4TSTAT|ENCFG4ISTAT|ENCFG4DATA|ENSAVEPTRS) SEQINTSRC[0x0] DFCNTRL[0x4]:(DIRECTION) DFSTATUS[0x89]:(FIFOEMP|HDONE|PRELOAD_AVAIL) SG_CACHE_SHADOW[0x2]:(LAST_SEG) SG_STATE[0x0] DFFSXFRCTL[0x0] SOFFCNT[0x0] MDFFSTAT[0x5]:(FIFOFREE|DLZERO) SHADDR = 0x00, SHCNT = 0x0 HADDR = 0x00, HCNT = 0x0 CCSGCTL[0x10]:(SG_CACHE_AVAIL) ahd0: FIFO1 Free, LONGJMP == 0x826d, SCB 0x6a SEQIMODE[0x3f]:(ENCFG4TCMD|ENCFG4ICMD|ENCFG4TSTAT|ENCFG4ISTAT|ENCFG4DATA|ENSAVEPTRS) SEQINTSRC[0x0] DFCNTRL[0x0] DFSTATUS[0x89]:(FIFOEMP|HDONE|PRELOAD_AVAIL) SG_CACHE_SHADOW[0x2]:(LAST_SEG) SG_STATE[0x0] DFFSXFRCTL[0x0] SOFFCNT[0x0] MDFFSTAT[0x5]:(FIFOFREE|DLZERO) SHADDR = 0x00, SHCNT = 0x0 HADDR = 0x00, HCNT = 0x0 CCSGCTL[0x10]:(SG_CACHE_AVAIL) LQIN: 0x8 0x0 0x0 0x4a 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 ahd0: LQISTATE = 0x1, LQOSTATE = 0x1a, OPTIONMODE = 0x42 ahd0: OS_SPACE_CNT = 0x1f MAXCMDCNT = 0x4d ahd0: SAVED_SCSIID = 0x0 SAVED_LUN = 0x0 SIMODE0[0xc]:(ENOVERRUN|ENIOERR) CCSCBCTL[0x4]:(CCSCBDIR) ahd0: REG0 == 0x4f, SINDEX = 0x102, DINDEX = 0x102 ahd0: SCBPTR == 0x4e, SCB_NEXT == 0x4f, SCB_NEXT2 == 0xff5d CDB 2a 0 d 1c 20 84 STACK: 0x24 0x0 0x0 0x0 0x0 0x0 0x0 0x0 <<<<<<<<<<<<<<<<< Dump Card State Ends >>>>>>>>>>>>>>>>>> (da0:ahd0:0:0:0): SCB 29 - timed out (da0:ahd0:0:0:0): Other SCB Timeout (da0:ahd0:0:0:0): No other SCB worth waiting for... ahd0: Issued Channel A Bus Reset. 82 SCBs aborted Copied 18 bytes of sense data offset 12: 0x70 0x0 0x6 0x0 0x0 0x0 0x0 0xa 0x0 0x0 0x0 0x0 0x29 0x2 0x2 0x0 0x0 0x0 (da0:ahd0:0:0:0): WRITE(10). CDB: 2a 0 1 dd fb e4 0 0 20 0 (da0:ahd0:0:0:0): CAM Status: SCSI Status Error (da0:ahd0:0:0:0): SCSI Status: Check Condition (da0:ahd0:0:0:0): UNIT ATTENTION asc:29,2 (da0:ahd0:0:0:0): Scsi bus reset occurred field replaceable unit: 2 (da0:ahd0:0:0:0): Retrying Command (per Sense Data) xxxxxxxxxxxxxxxx MESSAGES END xxxxxxxxxxxxxxxxx ----- Original Message ----- From: "Justin T. Gibbs" To: "Hutterer Robert" ; Sent: Thursday, August 18, 2005 3:44 AM Subject: Re: DELL SC430 & ahd0: >> My dell Sc430 Server with Freebsd 5.4 gives soon after a reboot and >> minimal >> harddisk actions (erase a file or directory) strange messages concerning >> Adaptec SCSI adapter: > >>From what I can tell from the full card dump state, the 39320 attempted > to send 77 transactions to your drive during a single connection. This > connection hung, and the timeout occurred. Since the drive controlls > the connection, it can cut the initiator off at any time if too many > commands are sent. So, this looks like a drive firmware bug. You > should contact Dell to find out if newer firmware is available for your > drive. In the mean time, you can try lowering the tag depth allowed > for your device. If the failure occurs sometime after rc processing, > you can make a call early in the transition to multi-user like so: > > camcontrol tags da0 -N 64 # or some lower number > > If that won't work for you, you can enter a quirk into sys/cam/cam_xpt.c > or just modify the last quirk entry (the default) to have a lower tag > depth (it is currently 255). > > If you do get new firmware for your drive and that resolves this issue, > please follow up to the list (and cc me if you don't mind) so that > others that encounter the same issue can benefit from your experience. > > -- > Justin >