From owner-freebsd-scsi@freebsd.org Fri Dec 9 14:23:45 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1C61C68486 for ; Fri, 9 Dec 2016 14:23:45 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 161B01AA for ; Fri, 9 Dec 2016 14:23:44 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id uB9ENfWU056333 for ; Fri, 9 Dec 2016 15:23:41 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 44F5FAE0; Fri, 9 Dec 2016 15:23:40 +0100 (CET) Message-ID: <584ABE6B.8060703@omnilan.de> Date: Fri, 09 Dec 2016 15:23:39 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Subject: sporadic CAM (all devices) outage on 11-stable, mps(4), ahci(4) and bhyve(8) involved. X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigF2CB364B72E040B5B6F2EB6E" X-Greylist: ACL 119 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Fri, 09 Dec 2016 15:23:41 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 14:23:46 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigF2CB364B72E040B5B6F2EB6E Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hello, I have big problems with my 11-stable bhyve playground. Suddenly (the machine ran without CAM-I/O-load for days), all CAM devices fail one after another =E2=80=93 reproducable. It happens e.g. during buildworld, in the following example buildworld run inside bhyve guest. The system has a zfs bootpool and root on mirror-stripe zpool. Unfortunately I only have the kernel messages, see next lines. I guess the interesting point is that _all_ devices over different controllers are affected. There's mps(4) and ahci(4) involved. Can someony with SCB/CAM/SCSIcmd understanding see what's wrong here? Thanks in advance, -Harry ahcich6: Timeout on slot 6 port 0 ahcich6: is 00000000 cs 00000000 ss 00003fc0 rs 00003fc0 tfd 40 serr 00000000 cmd 0004cc17 (ada6:ahcich6:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 c0 da 7f 40 06 00 00 00 00 00 (ada6:ahcich6:0:0:0): CAM status: Command timeout (ada6:ahcich6:0:0:0): Retrying command ahcich6: Timeout on slot 20 port 0 ahcich6: is 00000000 cs 0e000000 ss 0ff00000 rs 0ff00000 tfd c0 serr 00000000 cmd 0004d917 (ada6:ahcich6:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 c0 da 7f 40 06 00 00 00 00 00 (ada6:ahcich6:0:0:0): CAM status: Command timeout (ada6:ahcich6:0:0:0): Retrying command ahcich6: Timeout on slot 28 port 0 ahcich6: is 00000000 cs 10000000 ss 00000000 rs 10000000 tfd 150 serr 00000000 cmd 0004dc17 (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich6:0:0:0): CAM status: Command timeout (aprobe0:ahcich6:0:0:0): Retrying command ahcich6: Timeout on slot 29 port 0 ahcich6: is 00000000 cs 20000000 ss 00000000 rs 20000000 tfd 150 serr 00000000 cmd 0004dd17 (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich6:0:0:0): CAM status: Command timeout (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted ahcich6: Timeout on slot 30 port 0 ahcich6: is 00000000 cs 40000000 ss 00000000 rs 40000000 tfd 150 serr 00000000 cmd 0004de17 (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich6:0:0:0): CAM status: Command timeout (aprobe0:ahcich6:0:0:0): Error 5, Retry was blocked ada6 at ahcich6 bus 0 scbus12 target 0 lun 0 ada6: s/n BTMP317600EG060AGN detached ahcich6: Timeout on slot 31 port 0 ahcich6: is 00000000 cs 80000000 ss 00000000 rs 80000000 tfd 150 serr 00000000 cmd 0004df17 (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich6:0:0:0): CAM status: Command timeout (aprobe0:ahcich6:0:0:0): Retrying command ahcich6: Timeout on slot 0 port 0 ahcich6: is 00000000 cs 00000001 ss 00000000 rs 00000001 tfd 150 serr 00000000 cmd 0004c017 (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich6:0:0:0): CAM status: Command timeout (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted ahcich6: Poll timeout on slot 1 port 0 ahcich6: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr 00000000 cmd 0004c117 (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich6:0:0:0): CAM status: Command timeout (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted SGL for command 0xfffffe000112f750 0000 0c 00 00 00 00 00 00 00 00 00 00 00 7c d1 b9 bc |............|...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 SGL for command 0xfffffe000112c810 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error 0a(da0: 00mps0:0: 002: 000): 00Retrying command 00 00 00 00 00 00 00 0c ae b9 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 |*..A............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:mps0:0:4:0): Retrying command (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) SGL for command 0xfffffe000112ff30 (da0:0000 mps0:0: 0c2: 000): 00Retrying command (per sense data) 00 00 00 00 00 00 00 00 00 64 d7 b9 bc |............d...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): Retrying command (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) SGL for command 0xfffffe000112c030 0000 0c 00 00 00 00 00 00 00 00 00 00 00 24 a8 b9 bc |............$...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): Retrying command (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da2:mps0:0:4:0): Retrying command (per sense data) (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 = 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) SGL for command 0xfffffe00010fd800 0000 0a 00 00 00 00 00 00 00 00 00 00 00 00 7a b7 bc |.............z..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 SGL for command 0xfffffe00010f2ac0 (da2:mps0:0:4:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 = 00 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error 0a(da2: 00mps0:0: 004: 000): 00Retrying command 00 00 00 00 00 00 00 10 f8 b6 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 |*..A............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 SGL for command 0xfffffe0001105750 0000 0c 00 00 00 00 00 00 00 00 00 00 00 7c d9 b7 bc |............|...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 SGL for command 0xfffffe00010ed180 0000 0c 00 00 00 00 00 00 00 00 00 00 00 20 b5 b6 bc |............ ...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:mps0:0:4:0): Retrying command (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 = 00 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): Error 5, Retries exhausted (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): Error 5, Retries exhausted (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da2:mps0:0:4:0): Retrying command (per sense data) (da0:mps0:0:2:0): READ(10). CDB: 28 00 1b e2 9e 10 00 00 10 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) SGL for command 0xfffffe00010ea4e0 0000 0a 00 00 00 00 00 00 00 00 00 00 00 a8 93 b6 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 30 00 00 |.............0..| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 00 b9 a1 18 00 00 00 00 00 00 00 00 00 00 00 |................| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 16 00 30 bc be 00 00 00 00 00 10 00 96 |.....0..........| 0070 00 10 b7 bb 00 00 00 00 80 00 00 30 00 16 b2 bc |...........0....| seg0 flags=3D0x16 len=3D0x001000 addr=3D0x00000000bebc3000 seg1 flags=3D0x96 len=3D0x001000 addr=3D0x00000000bbb71000 chain flags=3D0x30 len=3D0x80 Offset=3D0x0 Address=3D0xbcb21600 0000 00 10 00 d7 00 70 29 a7 01 00 00 00 00 00 00 00 |.....p).........| 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| seg2 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001a7297000 SGL for command 0xfffffe000112d530 (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 b9 a1 18 00 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error 0a(da2: 00mps0:0: 004: 000): 00Retrying command 00 00 00 00 00 00 00 e4 b7 b9 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 |*..A............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:mps0:0:4:0): Error 5, Retries exhausted (da2:mps0:0:4:0): READ(6). CDB: 08 00 02 10 10 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da2:mps0:0:4:0): Retrying command (per sense data) SGL for command 0xfffffe000110a760 0000 0a 00 00 00 00 00 00 00 00 00 00 00 88 15 b8 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 |...n............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000e00 addr=3D0x00000001d0d81000 (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:mps0:0:4:0): Retrying command (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da2:mps0:0:4:0): Retrying command (per sense data) SGL for command 0xfffffe0001105b40 0000 0a 00 00 00 00 00 00 00 00 00 00 00 70 dc b7 bc |............p...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 SGL for command 0xfffffe0001108fc0 (da2:mps0:0:4:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 = 00 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error 0a(da2: 00mps0:0: 004: 000): 00Retrying command 00 00 00 00 00 00 00 d0 03 b8 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 |...n............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000e00 addr=3D0x00000001d0d81000 (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:mps0:0:4:0): Retrying command (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da2:mps0:0:4:0): Retrying command (per sense data) SGL for command 0xfffffe0001115350 0000 0a 00 00 00 00 00 00 00 00 00 00 00 7c 96 b8 bc |............|...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 |...n............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000e00 addr=3D0x00000001d0d81000 (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:mps0:0:4:0): Error 5, Retries exhausted (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 49 08 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da2:mps0:0:4:0): Retrying command (per sense data) (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 98 f4 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da2:mps0:0:4:0): Retrying command (per sense data) SGL for command 0xfffffe00010e5cb0 0000 0c 00 00 00 00 00 00 00 00 00 00 00 84 5d b6 bc |.............]..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 |*..._...........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 d0 b4 bb 00 00 00 00 00 00 00 00 00 00 00 00 |................| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): Retrying command (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) SGL for command 0xfffffe00010ff4e0 0000 0c 00 00 00 00 00 00 00 00 00 00 00 a8 8f b7 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 af d9 99 00 00 08 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x0000000012fc8000 SGL for command 0xfffffe0001130080 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error 0c(da0: 00mps0:0: 002: 000): 00Retrying command 00 00 00 00 00 00 00 60 d8 b9 bc |............`...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 |*..._ahcich6: =2ETimeout on slot 2 port 0 =2Eahcich6: .is 00000000 cs 00000004 ss 00000000 rs 00000004 tfd 150 serr= 00000000 cmd 0004c217 =2E.......| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 (ada6:ahcich6:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00 (ada6:ahcich6:0:0:0): CAM status: Command timeout (ada6:ahcich6:0:0:0): Error 5, Periph was invalidated ahcich6: Poll timeout on slot 3 port 0 ahcich6: is 00000000 cs 00000008 ss 00000000 rs 00000008 tfd 150 serr 00000000 cmd 0004c317 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): Retrying command (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich6:0:0:0): CAM status: Command timeout (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) SGL for command 0xfffffe0001125340 0000 0c 00 00 00 00 00 00 00 00 00 00 00 70 56 b9 bc |............pV..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 SGL for command 0xfffffe00010f9510 (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 = 00 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error 0c(da0: 00mps0:0: 002: 000): 00Retrying command 00 00 00 00 00 00 00 cc 47 b7 bc |.............G..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 |*..._...........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): Error 5, Retries exhausted (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 3d 00 00 0a 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 9e 00 00 03 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da0:mps0:0:2:0): Retrying command (per sense data) SGL for command 0xfffffe0001115dd0 0000 09 00 00 00 00 00 00 00 00 00 00 00 5c 9e b8 bc |............\134...= | 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 SGL for command 0xfffffe000112f8a0 (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error 0b(da3: 00mps0:0: 005: 000): 00Retrying command 00 00 00 00 00 00 00 78 d2 b9 bc |............x...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error (da1:mps0:0:3:0): Retrying command (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da3:mps0:0:5:0): CAM status: SCSI Status Error (da3:mps0:0:5:0): SCSI status: Check Condition (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da3:mps0:0:5:0): Retrying command (per sense data) (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da1:mps0:0:3:0): CAM status: SCSI Status Error (da1:mps0:0:3:0): SCSI status: Check Condition (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) SGL for command 0xfffffe000111ade0 (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) 00 00 00 00 00 00 00 00 00 68 da b8 bc |............h...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error (da3:mps0:0:5:0): Retrying command SGL for command 0xfffffe00011165b0 0000 0b 00 00 00 00 00 00 00 00 00 00 00 44 a4 b8 bc |............D...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 SGL for command 0xfffffe000112a9e0 (da1:mps0:0:3:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 = 00 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error 0b(da1: 00mps0:0: 003: 000): 00Retrying command 00 00 00 00 00 00 00 68 97 b9 bc |............h...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error (da1:mps0:0:3:0): Retrying command (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da3:mps0:0:5:0): CAM status: SCSI Status Error (da3:mps0:0:5:0): SCSI status: Check Condition (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da3:mps0:0:5:0): Retrying command (per sense data) (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da1:mps0:0:3:0): CAM status: SCSI Status Error (da1:mps0:0:3:0): SCSI status: Check Condition (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) SGL for command 0xfffffe000111a4b0 (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) 00 00 00 00 00 00 00 00 00 84 d3 b8 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error (da3:mps0:0:5:0): Error 5, Retries exhausted SGL for command 0xfffffe000112dbc0 0000 0b 00 00 00 00 00 00 00 00 00 00 00 d0 bc b9 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error (da1:mps0:0:3:0): Error 5, Retries exhausted (da3:mps0:0:5:0): WRITE(6). CDB: 0a 0c 0d a4 08 00 (da3:mps0:0:5:0): CAM status: SCSI Status Error (da3:mps0:0:5:0): SCSI status: Check Condition (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da3:mps0:0:5:0): Retrying command (per sense data) (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 af 8b 04 00 00 10 00 (da1:mps0:0:3:0): CAM status: SCSI Status Error (da1:mps0:0:3:0): SCSI status: Check Condition (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da1:mps0:0:3:0): Retrying command (per sense data) SGL for command 0xfffffe00010e1870 0000 0b 00 00 00 00 00 00 00 00 00 00 00 54 2a b6 bc |............T*..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 30 d5 76 01 00 00 00 80 00 00 30 80 13 b2 bc |.0.v.......0....| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 SGL for command 0xfffffe0001123a50 (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error 09(da1: 00mps0:0: 003: 000): 00Retrying command 00 00 00 00 00 00 00 bc 43 b9 bc |.............C..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 SGL for command 0xfffffe0001107040 0000 0c 00 00 00 00 00 00 00 00 00 00 00 30 ec b7 bc |............0...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 af da 71 00 00 08 00 00 00 00 00 00 00 |*....q..........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 |................| 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x0000000012fc8000 SGL for command 0xfffffe00010ea390 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da 71 00 00 08 00 0000 0c 00 00 00 00 00 00 00 00 00 00 00 ac 92 b6 bc |................| 0010 00(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error= 00(da0: fcmps0:0: 002: 180): 00Retrying command 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 SGL for command 0xfffffe000112e640 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error 0a(da0: 00mps0:0: 002: 00 00 00 00 00 00 00 00 00 b0 c4 b9 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| 0020 00 00 000): 00Retrying command 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 00 ba 71 08 00 00 00 00 00 00 00 00 00 00 00 |...q............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 |................| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x0000000012fc8000 SGL for command 0xfffffe00011312e0 (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 71 08 00 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error 0a(da2: 00mps0:0: 004: 000): 00Retrying command 00 00 00 00 00 00 00 28 e6 b9 bc |............(...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 |................| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:(da3:mps0:0:mps0:0:4:5:0): 0): Retrying command Retrying command (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da3:mps0:0:5:0): CAM status: SCSI Status Error (da3:mps0:0:5:0): SCSI status: Check Condition (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) (da3:mps0:0:5:0): Retrying command (per sense data) (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da1:mps0:0:3:0): CAM status: SCSI Status Error (da1:mps0:0:3:0): SCSI status: Check Condition (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) SGL for command 0xfffffe0001125b20 (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) 00 00 00 00 00 00 00 00 00 58 5c b9 bc |............X\134..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error (da3:mps0:0:5:0): Retrying command SGL for command 0xfffffe00010ed960 0000 0b 00 00 00 00 00 00 00 00 00 00 00 08 bb b6 bc |................| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 20 84 01 00 00 00 00 00 00 00 00 00 00 00 |.. .............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 71 08 00 (da1:(da2:mps0:0:4:0): CAM status: SCSI Status Error mps0:0:(da2:mps0:0:4:0): SCSI status: Check Condition 3:(da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) 0): (da2:Retrying command mps0:0:(da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 4:(da3:mps0:0:5:0): CAM status: SCSI Status Error 0): SGL for command 0xfffffe00010e1f00 (da3:mps0:0:5:0): SCSI status: Check Condition Retrying command (per sense data) 0000 (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) 0a(da3: 00mps0:0: 005:(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da 71 00 00 08 00 000): (da0:mps0:0:2:0): CAM status: SCSI Status Error 00(da0:mps0:0:2:0): SCSI status: Check Condition Retrying command (per sense data) 00 00 00 00 00 00 00 40 2f b6 bc |............@/..| 0010 00(da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) 00(da0: fcmps0:0: 002: 180): 00Retrying command (per sense data) 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 |................| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 SGL for command 0xfffffe000112b460 0000 0c 00 00 00 00 00 00 00 00 00 00 00 48 9f b9 bc |............H...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 (da1:mps0:0:3:0): CAM status: SCSI Status Error (da1:mps0:0:3:0): SCSI status: Check Condition (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) SGL for command 0xfffffe000111b710 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error (da1:(da2:0000 mps0:0:mps0:0: 093: 000): 4:Retrying command (per sense da= ta) 0): 00 00Retrying command 00 00 00 00 00 00 00 00 4c e1 b8 bc |............L...| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error (da3:(da0:mps0:0:mps0:0:5:2:0): 0): Error 5, Retries exhausted Retrying command (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 (da2:mps0:0:4:0): CAM status: SCSI Status Error (da2:mps0:0:4:0): SCSI status: Check Condition (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) SGL for command 0xfffffe00011272c0 (da2:0000 mps0:0: 0b4: 000): 00Retrying command (per sense data) 00 00 00 00 00 00 00 00 00 10 6e b9 bc |.............n..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 SGL for command 0xfffffe000110de80 (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error 0c(da1: 00mps0:0: 003: 000): 00Error 5, Retries exhausted 00 00 00 00 00 00 00 e0 3e b8 bc |.............>..| 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 |*...............| 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da 71 00 00 08 00 (da0:mps0:0:2:0): CAM status: SCSI Status Error (da0:mps0:0:2:0): SCSI status: Check Condition (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, or bus device reset occurred) SGL for command 0xfffffe0001126450 (da0:0000 mps0:0: 0a2: 000): 00Retrying command (per sense data) 00(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 00(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error 00(da0: 00 00 00 00 00 00 3c 63 b9 bc |............ Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00C49C68E90 for ; Fri, 9 Dec 2016 14:42:39 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wj0-x232.google.com (mail-wj0-x232.google.com [IPv6:2a00:1450:400c:c01::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 351B71227 for ; Fri, 9 Dec 2016 14:42:38 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wj0-x232.google.com with SMTP id xy5so16374698wjc.0 for ; Fri, 09 Dec 2016 06:42:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=5dKd2YNEp4jyPTLOs4cYZ9A7hIf6TSbIqmD+viVspCM=; b=vTKPypfCe5B8oK5LA9dO6onpuKr8jMLDr6xuN2R6bT6Ok71kR+XmOgyGLPjTIUmVyU QOBG9AW91kxUcXKB8JEZuEZCU+5eyx/pxxF1dxYez98RDhqOJ9Uizb3lL/qRE9QLbZLD pMa8FzkqH50DbwZzNuYB7qJQ5l3Wvigzodltlmq8WJBixQTV2tuZ589lC2c80/SJ5Seg hSh+1jWxRMvAeueFsJTeF6rx9Onfgj03raUSo/n+qi1T/ycmfEB5VQl6mnpG26+Aioy4 9jk9OPlQv7vwA199yGd3mjTlWx698+oo/3DiB6v5w3euQGeMz3hzjiH+1/BtDvl59nlS z2sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=5dKd2YNEp4jyPTLOs4cYZ9A7hIf6TSbIqmD+viVspCM=; b=eG6D5tkXCdppxjtwXqomP7XHzbsrlMhKslBaF/Q0PjOsgIJzVTviJ+cLEBN2uz5bhE ypIsOwLznCsTtj0dvJpEXrH2ygrfY67TsF2uuEt9LncMzxGleSJ1UQsXwmcSzhAt7b9g 0WmRGKAuGx4d/GqlidRfEnmmJ0/vDimQJo9mIOimG45jfe9iku8/aSJ0OCykpPxuO9aP Dmsu3noNbJGvyM0+RkwVqdgE0wyeE8r4jCVbtsRMfCfoKQIyZ+aHOZlTdTuDHo5r1K4F vd7M85PhANyQpCmtiQqY6NdPc4BSG71SmqcUzqbsitHViefe9EbY77NhP3c1E6dZSl4t GJhA== X-Gm-Message-State: AKaTC00yi2D9yjTQa6IVBSHlPLMv07Pwa05kyEvhTMR90kG66lwNa12h2HB6VhU1uPBZi5WR X-Received: by 10.194.115.226 with SMTP id jr2mr67562858wjb.75.1481294554200; Fri, 09 Dec 2016 06:42:34 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id b15sm20946300wma.5.2016.12.09.06.42.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Dec 2016 06:42:33 -0800 (PST) Subject: Re: sporadic CAM (all devices) outage on 11-stable, mps(4), ahci(4) and bhyve(8) involved. To: freebsd-scsi@freebsd.org References: <584ABE6B.8060703@omnilan.de> From: Steven Hartland Message-ID: Date: Fri, 9 Dec 2016 14:43:07 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <584ABE6B.8060703@omnilan.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 14:42:39 -0000 Bad memory or CPU? On 09/12/2016 14:23, Harald Schmalzbauer wrote: > Hello, > > I have big problems with my 11-stable bhyve playground. > > Suddenly (the machine ran without CAM-I/O-load for days), all CAM > devices fail one after another – reproducable. > It happens e.g. during buildworld, in the following example buildworld > run inside bhyve guest. > > The system has a zfs bootpool and root on mirror-stripe zpool. > Unfortunately I only have the kernel messages, see next lines. > > I guess the interesting point is that _all_ devices over different > controllers are affected. There's mps(4) and ahci(4) involved. > > Can someony with SCB/CAM/SCSIcmd understanding see what's wrong here? > > Thanks in advance, > > -Harry > > ahcich6: Timeout on slot 6 port 0 > ahcich6: is 00000000 cs 00000000 ss 00003fc0 rs 00003fc0 tfd 40 serr > 00000000 cmd 0004cc17 > (ada6:ahcich6:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 c0 da 7f 40 06 00 > 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 20 port 0 > ahcich6: is 00000000 cs 0e000000 ss 0ff00000 rs 0ff00000 tfd c0 serr > 00000000 cmd 0004d917 > (ada6:ahcich6:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 c0 da 7f 40 06 00 > 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 28 port 0 > ahcich6: is 00000000 cs 10000000 ss 00000000 rs 10000000 tfd 150 serr > 00000000 cmd 0004dc17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 29 port 0 > ahcich6: is 00000000 cs 20000000 ss 00000000 rs 20000000 tfd 150 serr > 00000000 cmd 0004dd17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > ahcich6: Timeout on slot 30 port 0 > ahcich6: is 00000000 cs 40000000 ss 00000000 rs 40000000 tfd 150 serr > 00000000 cmd 0004de17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retry was blocked > ada6 at ahcich6 bus 0 scbus12 target 0 lun 0 > ada6: s/n BTMP317600EG060AGN detached > ahcich6: Timeout on slot 31 port 0 > ahcich6: is 00000000 cs 80000000 ss 00000000 rs 80000000 tfd 150 serr > 00000000 cmd 0004df17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 0 port 0 > ahcich6: is 00000000 cs 00000001 ss 00000000 rs 00000001 tfd 150 serr > 00000000 cmd 0004c017 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > ahcich6: Poll timeout on slot 1 port 0 > ahcich6: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr > 00000000 cmd 0004c117 > (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > SGL for command 0xfffffe000112f750 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 7c d1 b9 bc |............|...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0d81000 > SGL for command 0xfffffe000112c810 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > 0a(da0: 00mps0:0: 002: 000): 00Retrying command > 00 00 00 00 00 00 00 0c ae b9 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 |*..A............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0709000 > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe000112ff30 > (da0:0000 mps0:0: 0c2: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 64 d7 b9 bc |............d...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0d81000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe000112c030 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 24 a8 b9 bc |............$...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0d81000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe00010fd800 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 00 7a b7 bc |.............z..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000000 addr=0x0000000000000000 > SGL for command 0xfffffe00010f2ac0 > (da2:mps0:0:4:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 10 f8 b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 |*..A............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0709000 > SGL for command 0xfffffe0001105750 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 7c d9 b7 bc |............|...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000000 addr=0x0000000000000000 > SGL for command 0xfffffe00010ed180 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 20 b5 b6 bc |............ ...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Error 5, Retries exhausted > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Error 5, Retries exhausted > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > (da0:mps0:0:2:0): READ(10). CDB: 28 00 1b e2 9e 10 00 00 10 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe00010ea4e0 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 a8 93 b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 30 00 00 |.............0..| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 00 b9 a1 18 00 00 00 00 00 00 00 00 00 00 00 |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 16 00 30 bc be 00 00 00 00 00 10 00 96 |.....0..........| > 0070 00 10 b7 bb 00 00 00 00 80 00 00 30 00 16 b2 bc |...........0....| > seg0 flags=0x16 len=0x001000 addr=0x00000000bebc3000 > seg1 flags=0x96 len=0x001000 addr=0x00000000bbb71000 > chain flags=0x30 len=0x80 Offset=0x0 Address=0xbcb21600 > 0000 00 10 00 d7 00 70 29 a7 01 00 00 00 00 00 00 00 |.....p).........| > 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > seg2 flags=0xd7 len=0x001000 addr=0x00000001a7297000 > SGL for command 0xfffffe000112d530 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 b9 a1 18 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 e4 b7 b9 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 |*..A............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0709000 > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Error 5, Retries exhausted > (da2:mps0:0:4:0): READ(6). CDB: 08 00 02 10 10 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe000110a760 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 88 15 b8 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 |...n............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000e00 addr=0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe0001105b40 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 70 dc b7 bc |............p...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000000 addr=0x0000000000000000 > SGL for command 0xfffffe0001108fc0 > (da2:mps0:0:4:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 d0 03 b8 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 |...n............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000e00 addr=0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe0001115350 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 7c 96 b8 bc |............|...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 |...n............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000e00 addr=0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Error 5, Retries exhausted > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 49 08 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 98 f4 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe00010e5cb0 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 84 5d b6 bc |.............]..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 |*..._...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 d0 b4 bb 00 00 00 00 00 00 00 00 00 00 00 00 |................| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0709000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe00010ff4e0 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 a8 8f b7 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 af d9 99 00 00 08 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x001000 addr=0x0000000012fc8000 > SGL for command 0xfffffe0001130080 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > 0c(da0: 00mps0:0: 002: 000): 00Retrying command > 00 00 00 00 00 00 00 60 d8 b9 bc |............`...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 |*..._ahcich6: > .Timeout on slot 2 port 0 > .ahcich6: .is 00000000 cs 00000004 ss 00000000 rs 00000004 tfd 150 serr > 00000000 cmd 0004c217 > ........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0709000 > (ada6:ahcich6:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 > 00 00 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Error 5, Periph was invalidated > ahcich6: Poll timeout on slot 3 port 0 > ahcich6: is 00000000 cs 00000008 ss 00000000 rs 00000008 tfd 150 serr > 00000000 cmd 0004c317 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe0001125340 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 70 56 b9 bc |............pV..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000000 addr=0x0000000000000000 > SGL for command 0xfffffe00010f9510 > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > 0c(da0: 00mps0:0: 002: 000): 00Retrying command > 00 00 00 00 00 00 00 cc 47 b7 bc |.............G..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 |*..._...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x001000 addr=0x00000001d0709000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Error 5, Retries exhausted > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 3d 00 00 0a 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 9e 00 00 03 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe0001115dd0 > 0000 09 00 00 00 00 00 00 00 00 00 00 00 5c 9e b8 bc |............\134...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > SGL for command 0xfffffe000112f8a0 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > 0b(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 78 d2 b9 bc |............x...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe000111ade0 > (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 68 da b8 bc |............h...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > SGL for command 0xfffffe00011165b0 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 44 a4 b8 bc |............D...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000000 addr=0x0000000000000000 > SGL for command 0xfffffe000112a9e0 > (da1:mps0:0:3:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > 0b(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 68 97 b9 bc |............h...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe000111a4b0 > (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 84 d3 b8 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Error 5, Retries exhausted > SGL for command 0xfffffe000112dbc0 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 d0 bc b9 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Error 5, Retries exhausted > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 0c 0d a4 08 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 af 8b 04 00 00 10 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da1:mps0:0:3:0): Retrying command (per sense data) > SGL for command 0xfffffe00010e1870 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 54 2a b6 bc |............T*..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 30 d5 76 01 00 00 00 80 00 00 30 80 13 b2 bc |.0.v.......0....| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > SGL for command 0xfffffe0001123a50 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > 09(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 bc 43 b9 bc |.............C..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > SGL for command 0xfffffe0001107040 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 30 ec b7 bc |............0...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 af da 71 00 00 08 00 00 00 00 00 00 00 |*....q..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 |................| > 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > seg0 flags=0xd7 len=0x001000 addr=0x0000000012fc8000 > SGL for command 0xfffffe00010ea390 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da 71 00 00 08 00 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 ac 92 b6 bc |................| > 0010 00(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > 00(da0: fcmps0:0: 002: 180): 00Retrying command > 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > SGL for command 0xfffffe000112e640 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > 0a(da0: 00mps0:0: 002: 00 00 00 00 00 00 00 00 00 b0 c4 b9 bc > |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 |................| > 0020 00 00 000): 00Retrying command > 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 00 ba 71 08 00 00 00 00 00 00 00 00 00 00 00 |...q............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x001000 addr=0x0000000012fc8000 > SGL for command 0xfffffe00011312e0 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 71 08 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 28 e6 b9 bc |............(...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:(da3:mps0:0:mps0:0:4:5:0): 0): Retrying command > Retrying command > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe0001125b20 > (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 58 5c b9 bc |............X\134..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > SGL for command 0xfffffe00010ed960 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 08 bb b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 20 84 01 00 00 00 00 00 00 00 00 00 00 00 |.. .............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 71 08 00 > (da1:(da2:mps0:0:4:0): CAM status: SCSI Status Error > mps0:0:(da2:mps0:0:4:0): SCSI status: Check Condition > 3:(da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 0): (da2:Retrying command > mps0:0:(da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > 4:(da3:mps0:0:5:0): CAM status: SCSI Status Error > 0): SGL for command 0xfffffe00010e1f00 > (da3:mps0:0:5:0): SCSI status: Check Condition > Retrying command (per sense data) > 0000 (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 0a(da3: 00mps0:0: 005:(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da > 71 00 00 08 00 > 000): (da0:mps0:0:2:0): CAM status: SCSI Status Error > 00(da0:mps0:0:2:0): SCSI status: Check Condition > Retrying command (per sense data) > 00 00 00 00 00 00 00 40 2f b6 bc |............@/..| > 0010 00(da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 00(da0: fcmps0:0: 002: 180): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > SGL for command 0xfffffe000112b460 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 48 9f b9 bc |............H...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe000111b710 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:(da2:0000 mps0:0:mps0:0: 093: 000): 4:Retrying command (per sense data) > 0): 00 00Retrying command > 00 00 00 00 00 00 00 00 4c e1 b8 bc |............L...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:(da0:mps0:0:mps0:0:5:2:0): 0): Error 5, Retries exhausted > Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe00011272c0 > (da2:0000 mps0:0: 0b4: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 10 6e b9 bc |.............n..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > SGL for command 0xfffffe000110de80 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > 0c(da1: 00mps0:0: 003: 000): 00Error 5, Retries exhausted > 00 00 00 00 00 00 00 e0 3e b8 bc |.............>..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da 71 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe0001126450 > (da0:0000 mps0:0: 0a2: 000): 00Retrying command (per sense data) > 00(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > 00(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > 00(da0: 00 00 00 00 00 00 3c 63 b9 bc |............ 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00mps0:0: 0a2: 000): 00Retrying > command > |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > SGL for command 0xfffffe00010e4660 > (da3:0000 mps0:0: 0b5: 000): 00Retrying command (per sense data) > 00(da1:mps0:0:3:0): READ(10). CDB: 28 00 1b e2 9e 10 00 00 10 00 > 00(da1:mps0:0:3:0): CAM status: SCSI Status Error > 00(da1:mps0:0:3:0): SCSI status: Check Condition > 00(da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 00(da1: 00mps0:0: 003: 000): 00Retrying command (per sense data) > c8 4c b6 bc |.............L..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 |................| > 0040 28 00 00 af 22 10 00 00 10 00 00 00 00 00 00 00 |(..."...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 12 00 b0 87 88 01 00 00 00 00 10 00 d3 |................| > 0070 00 e0 41 a9 01 00 00 00 00 00 00 00 00 00 00 00 |..A.............| > seg0 flags=0x12 len=0x001000 addr=0x000000018887b000 > seg1 flags=0xd3 len=0x001000 addr=0x00000001a941e000 > SGL for command 0xfffffe00010eaa20 > (da1:mps0:0:3:0): READ(10). CDB: 28 00 00 af 22 10 00 00 10 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > 0b(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 98 97 b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 |................| > 0040 28 00 1b e2 9c 10 00 00 10 00 00 00 00 00 00 00 |(...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 10 00 12 00 30 b3 b2 01 00 00 00 00 10 00 d3 |.....0..........| > 0070 00 d0 b0 b2 01 00 00 00 00 00 00 00 00 00 00 00 |................| > seg0 flags=0x12 len=0x001000 addr=0x00000001b2b33000 > seg1 flags=0xd3 len=0x001000 addr=0x00000001b2b0d000 > SGL for command 0xfffffe00011211a0 > (da1:mps0:0:3:0): READ(10). CDB: 28 00 1b e2 9c 10 00 00 10 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > 0b(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 38 25 b9 bc |............8%..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 f0 5d 20 00 00 05 00 00 00 00 00 00 00 |*...] ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > SGL for command 0xfffffe000110b720 > (da2:mps0:0:4:0): SCSI status: Check Condition > 0000 (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 09(da2: 00mps0:0: 004: 000): 00Error 6, Retries exhausted > 00(da2: 00mps0:0: 004: 000): 00Invalidating pack > 00(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > 00(da0:mps0:0:2:0): CAM status: SCSI Status Error > 58(da0:mps0:0:2:0): SCSI status: Check Condition > 21(da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > b8(da0: bcmps0:0: |2:.0): .Retrying command (per sense data) > ..........X!..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 1b f2 42 10 00 00 10 00 00 00 00 00 00 00 |*...B...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 20 00 d7 00 f0 6c 09 00 00 00 00 00 00 00 00 |. ....l.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x002000 addr=0x00000000096cf000 > SGL for command 0xfffffe00010ef100 > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 1b f2 42 10 00 00 10 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 c0 cc b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 1b f2 40 10 00 00 10 00 00 00 00 00 00 00 |*...@...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 20 00 d7 00 40 8f 0c 00 00 00 00 00 00 00 00 |. ...@..........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x002000 addr=0x000000000c8f4000 > SGL for command 0xfffffe0001122a90 > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 1b f2 40 10 00 00 10 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 ec 37 b9 bc |.............7..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 41 3d 20 00 00 05 00 00 00 00 00 00 00 |*..A= ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > SGL for command 0xfffffe00010e11e0 > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 68 25 b6 bc |............h%..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. ..| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 00 02 10 10 00 00 00 00 00 00 00 00 00 00 00 |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 20 00 d7 00 70 bf 0c 00 00 00 00 00 00 00 00 |. ...p..........| > 0070 00 00 a0 88 01 00 00 00 00 00 00 00 00 00 00 00 |................| > seg0 flags=0xd7 len=0x002000 addr=0x000000000cbf7000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 00 02 10 10 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > SGL for command 0xfffffe0001126ed0 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > 0000 (da1:mps0:0:3:0): SCSI status: Check Condition > 0c(da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 00(da1: 00mps0:0: 003: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 1c 6b b9 bc |.............k..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000000 addr=0x0000000000000000 > SGL for command 0xfffffe0001122010 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 0c 30 b9 bc |.............0..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:(da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > 2:(da1:mps0:0:3:0): CAM status: SCSI Status Error > 0): (da1:mps0:0:3:0): SCSI status: Check Condition > Retrying command > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > (da1:(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > mps0:0:(da0:3:mps0:0:0): 2:Retrying command (per sense data) > 0): Error 5, Retries exhausted > SGL for command 0xfffffe00011209c0 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 50 1f b9 bc |............P...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 f0 5d 20 00 00 05 00 00 00 00 00 00 00 |*...] ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 00 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da3:(da0:mps0:0:mps0:0:5:2:0): 0): Retrying command (per sense data) > Error 6, Retries exhausted > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > (da0:(da1:mps0:0:3:0): CAM status: SCSI Status Error > mps0:0:(da1:mps0:0:3:0): SCSI status: Check Condition > 2:(da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 0): (da1:Invalidating pack > mps0:0:3:0): Error 6, Retries exhausted > (da1:mps0:0:3:0): Invalidating pack > SGL for command 0xfffffe000112de60 > 0000 09 00 00 00 00 00 00 00 00 00 00 00 c8 be b9 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 90 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 00 6b 4c 48 00 00 00 00 00 00 00 00 00 00 00 |..kLH...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 90 00 d7 00 60 cf 16 00 00 00 00 00 00 00 00 |.....`..........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x009000 addr=0x0000000016cf6000 > SGL for command 0xfffffe0001117420 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 00 6b 4c 48 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 18 af b8 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 50 01 00 |.............P..| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 0a 00 c5 98 a8 00 00 00 00 00 00 00 00 00 00 00 |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 50 01 d7 00 f0 3d 84 01 00 00 00 00 00 00 00 |.P....=.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 |..`.............| > seg0 flags=0xd7 len=0x015000 addr=0x00000001843df000 > SGL for command 0xfffffe00010f19b0 > 0000 09 00 00 00 00 00 00 00 00 00 00 00 44 eb b6 bc |............D...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 |................| > 0040 2a 00 05 41 3d 20 00 00 05 00 00 00 00 00 00 00 |*..A= ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 |..@.............| > seg0 flags=0xd7 len=0x000a00 addr=0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 00 c5 98 a8 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Error 6, Retries exhausted > (da3:mps0:0:5:0): Invalidating pack > pid 6760 (bhyve), uid 0, was killed: out of swap space > pid 1327 (ntpd), uid 0, was killed: out of swap space > ahcich6: Timeout on slot 4 port 0 > ahcich6: is 00000000 cs 00000010 ss 00000000 rs 00000010 tfd 150 serr > 00000000 cmd 0004c417 > (ada6:ahcich6:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 > 00 00 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Error 5, Periph was invalidated > ahcich6: Poll timeout on slot 5 port 0 > ahcich6: is 00000000 cs 00000020 ss 00000000 rs 00000020 tfd 150 serr > 00000000 cmd 0004c517 > (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > From owner-freebsd-scsi@freebsd.org Fri Dec 9 14:57:01 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B623C6D48D for ; Fri, 9 Dec 2016 14:57:01 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A875D1C7F for ; Fri, 9 Dec 2016 14:57:00 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id uB9Euwac056688; Fri, 9 Dec 2016 15:56:58 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id A817DAF1; Fri, 9 Dec 2016 15:56:58 +0100 (CET) Message-ID: <584AC63A.60301@omnilan.de> Date: Fri, 09 Dec 2016 15:56:58 +0100 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Steven Hartland CC: freebsd-scsi@freebsd.org Subject: Re: sporadic CAM (all devices) outage on 11-stable, mps(4), ahci(4) and bhyve(8) involved. References: <584ABE6B.8060703@omnilan.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Fri, 09 Dec 2016 15:56:58 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 14:57:01 -0000 Bezüglich Steven Hartland's Nachricht vom 09.12.2016 15:43 (localtime): > Bad memory or CPU? Quiet sure that this can be ruled out. It's a IvyBridge Xeon and ECC memory and neither BMC nor the OS registrerd any MCE. And, the machine is still operational (from cache, true for both, guest and host) until something from disk is needed... The machine doesn't show any problems without heavy bhyve-ahci-load. I'll see if changing ahci, to virtio-blk changes anything. At least for this other still (r309605) present odditiy https://lists.freebsd.org/pipermail/freebsd-stable/2016-October/086157.html it doesn't make a difference, while I found virtio-blk beeing some orders of magnitude faster for unlinking UFS files - most likely due to lacking TRIM supprt. But on bare metal deleting a object tree on a SSD (with enabled TRIM support) doesn't take 10 minutes... Thanks, -Harry From owner-freebsd-scsi@freebsd.org Fri Dec 9 15:34:16 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A101C6E09E for ; Fri, 9 Dec 2016 15:34:16 +0000 (UTC) (envelope-from kevin@your.org) Received: from mail.your.org (mail.your.org [IPv6:2001:4978:1:2::cc09:3717]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4721F14CC for ; Fri, 9 Dec 2016 15:34:16 +0000 (UTC) (envelope-from kevin@your.org) Received: from mail.your.org (chi02.mail.your.org [204.9.55.23]) by mail.your.org (Postfix) with ESMTP id 56A3528753C; Fri, 9 Dec 2016 15:34:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=your.org; h=content-type :mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s= selector1; bh=nXuuX1S1F0M3IkaN1E+gyNSZye4=; b=ZR1UUuzHMXVgVr8ktY OL706tYh05rc/Fmzt4vachWEQbMDqVxw25qGIRWgFogKFCTf1jzCMc2yG4sVgkMp YCut61IHU9yj4CkaPidF/g+C44OZof6ObvZb2WrLH/f5Sg3OkbftCwYgfVeLNqqD 4V+t6Er7S1MjhxRJsf1oDOIE8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=your.org; h=content-type :mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; q=dns; s= selector1; b=AtYrIjyfG9afgNMZwKJHfrGOnovKsJk7tC/oDL15e4SjRktAUdf Gfeo2miBDXpUmw2QJaHjnqAfjtM5XSVfxtPE6JtmZ/Ida6Autj27dzdaMpowf+rL 5NObhHkK/Ig6BfrURWdavQgKxSpyUS6htWRychJ70B/uIxxgQ8QslqLo= Received: from unassigned.v6.your.org (unknown [IPv6:2001:4978:1:45:18b1:ccd7:7e56:465a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.your.org (Postfix) with ESMTPSA id 3161828753B; Fri, 9 Dec 2016 15:34:15 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) Subject: Re: sporadic CAM (all devices) outage on 11-stable, mps(4), ahci(4) and bhyve(8) involved. From: Kevin Day In-Reply-To: <584ABE6B.8060703@omnilan.de> Date: Fri, 9 Dec 2016 09:34:14 -0600 Cc: freebsd-scsi@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <584ABE6B.8060703@omnilan.de> To: Harald Schmalzbauer X-Mailer: Apple Mail (2.3251) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 15:34:16 -0000 I=E2=80=99m not sure if this is related, but after upgrading from 10.x = to 11.0-RELEASE we=E2=80=99re seeing on multiple systems that = =E2=80=9Ccamcontrol rescan all=E2=80=9D causes all devices to detach in = a similar fashion. All mps and achi drives timeout and/or just = immediately disappear, then reattach. Interestingly mpr backed drives do = not disappear. > On Dec 9, 2016, at 8:23 AM, Harald Schmalzbauer = wrote: >=20 > Hello, >=20 > I have big problems with my 11-stable bhyve playground. >=20 > Suddenly (the machine ran without CAM-I/O-load for days), all CAM > devices fail one after another =E2=80=93 reproducable. > It happens e.g. during buildworld, in the following example buildworld > run inside bhyve guest. >=20 > The system has a zfs bootpool and root on mirror-stripe zpool. > Unfortunately I only have the kernel messages, see next lines. >=20 > I guess the interesting point is that _all_ devices over different > controllers are affected. There's mps(4) and ahci(4) involved. >=20 > Can someony with SCB/CAM/SCSIcmd understanding see what's wrong here? >=20 > Thanks in advance, >=20 > -Harry >=20 > ahcich6: Timeout on slot 6 port 0 > ahcich6: is 00000000 cs 00000000 ss 00003fc0 rs 00003fc0 tfd 40 serr > 00000000 cmd 0004cc17 > (ada6:ahcich6:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 c0 da 7f 40 06 00 > 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 20 port 0 > ahcich6: is 00000000 cs 0e000000 ss 0ff00000 rs 0ff00000 tfd c0 serr > 00000000 cmd 0004d917 > (ada6:ahcich6:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 c0 da 7f 40 06 00 > 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 28 port 0 > ahcich6: is 00000000 cs 10000000 ss 00000000 rs 10000000 tfd 150 serr > 00000000 cmd 0004dc17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 29 port 0 > ahcich6: is 00000000 cs 20000000 ss 00000000 rs 20000000 tfd 150 serr > 00000000 cmd 0004dd17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > ahcich6: Timeout on slot 30 port 0 > ahcich6: is 00000000 cs 40000000 ss 00000000 rs 40000000 tfd 150 serr > 00000000 cmd 0004de17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retry was blocked > ada6 at ahcich6 bus 0 scbus12 target 0 lun 0 > ada6: s/n BTMP317600EG060AGN detached > ahcich6: Timeout on slot 31 port 0 > ahcich6: is 00000000 cs 80000000 ss 00000000 rs 80000000 tfd 150 serr > 00000000 cmd 0004df17 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Retrying command > ahcich6: Timeout on slot 0 port 0 > ahcich6: is 00000000 cs 00000001 ss 00000000 rs 00000001 tfd 150 serr > 00000000 cmd 0004c017 > (aprobe0:ahcich6:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > ahcich6: Poll timeout on slot 1 port 0 > ahcich6: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr > 00000000 cmd 0004c117 > (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 = 00 > 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > SGL for command 0xfffffe000112f750 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 7c d1 b9 bc = |............|...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 > SGL for command 0xfffffe000112c810 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0a(da0: 00mps0:0: 002: 000): 00Retrying command > 00 00 00 00 00 00 00 0c ae b9 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 = |*..A............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe000112ff30 > (da0:0000 mps0:0: 0c2: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 64 d7 b9 bc |............d...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe000112c030 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 24 a8 b9 bc = |............$...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 = |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe00010fd800 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 00 7a b7 bc = |.............z..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 = |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 > SGL for command 0xfffffe00010f2ac0 > (da2:mps0:0:4:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 10 f8 b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 = |*..A............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 > SGL for command 0xfffffe0001105750 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 7c d9 b7 bc = |............|...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 = |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 > SGL for command 0xfffffe00010ed180 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 20 b5 b6 bc |............ = ...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 f0 ce ed 00 00 08 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 = |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Error 5, Retries exhausted > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 f0 ce ed 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Error 5, Retries exhausted > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > (da0:mps0:0:2:0): READ(10). CDB: 28 00 1b e2 9e 10 00 00 10 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe00010ea4e0 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 a8 93 b6 bc = |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 30 00 00 = |.............0..| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 00 b9 a1 18 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 16 00 30 bc be 00 00 00 00 00 10 00 96 = |.....0..........| > 0070 00 10 b7 bb 00 00 00 00 80 00 00 30 00 16 b2 bc = |...........0....| > seg0 flags=3D0x16 len=3D0x001000 addr=3D0x00000000bebc3000 > seg1 flags=3D0x96 len=3D0x001000 addr=3D0x00000000bbb71000 > chain flags=3D0x30 len=3D0x80 Offset=3D0x0 Address=3D0xbcb21600 > 0000 00 10 00 d7 00 70 29 a7 01 00 00 00 00 00 00 00 = |.....p).........| > 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0040 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > seg2 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001a7297000 > SGL for command 0xfffffe000112d530 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 b9 a1 18 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 e4 b7 b9 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 41 ae ed 00 00 08 00 00 00 00 00 00 00 = |*..A............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 > (da2:mps0:0:4:0): WRITE(10). CDB: 2a 00 05 41 ae ed 00 00 08 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Error 5, Retries exhausted > (da2:mps0:0:4:0): READ(6). CDB: 08 00 02 10 10 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe000110a760 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 88 15 b8 bc = |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 = |...n............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000e00 addr=3D0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe0001105b40 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 70 dc b7 bc = |............p...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 = |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 > SGL for command 0xfffffe0001108fc0 > (da2:mps0:0:4:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 d0 03 b8 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 = |...n............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000e00 addr=3D0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe0001115350 > 0000 0a 00 00 00 00 00 00 00 00 00 00 00 7c 96 b8 bc = |............|...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0e 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 1e e3 6e 07 00 00 00 00 00 00 00 00 00 00 00 = |...n............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0e 00 d7 00 10 d8 d0 01 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000e00 addr=3D0x00000001d0d81000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 6e 07 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Error 5, Retries exhausted > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 49 08 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e3 98 f4 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da2:mps0:0:4:0): Retrying command (per sense data) > SGL for command 0xfffffe00010e5cb0 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 84 5d b6 bc = |.............]..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 = |*..._...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 d0 b4 bb 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe00010ff4e0 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 a8 8f b7 bc = |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 af d9 99 00 00 08 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x0000000012fc8000 > SGL for command 0xfffffe0001130080 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0c(da0: 00mps0:0: 002: 000): 00Retrying command > 00 00 00 00 00 00 00 60 d8 b9 bc |............`...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 |*..._ahcich6: > .Timeout on slot 2 port 0 > .ahcich6: .is 00000000 cs 00000004 ss 00000000 rs 00000004 tfd 150 = serr > 00000000 cmd 0004c217 > ........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 > (ada6:ahcich6:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 = 40 > 00 00 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Error 5, Periph was invalidated > ahcich6: Poll timeout on slot 3 port 0 > ahcich6: is 00000000 cs 00000008 ss 00000000 rs 00000008 tfd 150 serr > 00000000 cmd 0004c317 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Retrying command > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af d9 99 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 = 00 > 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe0001125340 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 70 56 b9 bc = |............pV..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 = |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 > SGL for command 0xfffffe00010f9510 > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0c(da0: 00mps0:0: 002: 000): 00Retrying command > 00 00 00 00 00 00 00 cc 47 b7 bc |.............G..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 ef 5f 0b 00 00 08 00 00 00 00 00 00 00 = |*..._...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 90 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x00000001d0709000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 05 ef 5f 0b 00 00 08 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): Error 5, Retries exhausted > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 3d 00 00 0a 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 9e 00 00 03 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): Retrying command (per sense data) > SGL for command 0xfffffe0001115dd0 > 0000 09 00 00 00 00 00 00 00 00 00 00 00 5c 9e b8 bc = |............\134...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 = |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > SGL for command 0xfffffe000112f8a0 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0b(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 78 d2 b9 bc |............x...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 = |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe000111ade0 > (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 68 da b8 bc |............h...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 = |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > SGL for command 0xfffffe00011165b0 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 44 a4 b8 bc = |............D...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 = |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 > SGL for command 0xfffffe000112a9e0 > (da1:mps0:0:3:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0b(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 68 97 b9 bc |............h...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 = |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe000111a4b0 > (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 84 d3 b8 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 = |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Error 5, Retries exhausted > SGL for command 0xfffffe000112dbc0 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 d0 bc b9 bc = |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 = |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Error 5, Retries exhausted > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 0c 0d a4 08 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 af 8b 04 00 00 10 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da1:mps0:0:3:0): Retrying command (per sense data) > SGL for command 0xfffffe00010e1870 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 54 2a b6 bc = |............T*..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 = |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 30 d5 76 01 00 00 00 80 00 00 30 80 13 b2 bc = |.0.v.......0....| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > SGL for command 0xfffffe0001123a50 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter = Error > 09(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 bc 43 b9 bc |.............C..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 = |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > SGL for command 0xfffffe0001107040 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 30 ec b7 bc = |............0...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 af da 71 00 00 08 00 00 00 00 00 00 00 = |*....q..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x0000000012fc8000 > SGL for command 0xfffffe00010ea390 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da 71 00 00 08 00 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 ac 92 b6 bc = |................| > 0010 00(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter = Error > 00(da0: fcmps0:0: 002: 180): 00Retrying command > 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > SGL for command 0xfffffe000112e640 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > 0000 (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0a(da0: 00mps0:0: 002: 00 00 00 00 00 00 00 00 00 b0 c4 b9 bc > |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 10 00 00 = |................| > 0020 00 00 000): 00Retrying command > 06 00 00 00 00 00 00 00 00 00 00 00 |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 00 ba 71 08 00 00 00 00 00 00 00 00 00 00 00 = |...q............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 d7 00 80 fc 12 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x001000 addr=3D0x0000000012fc8000 > SGL for command 0xfffffe00011312e0 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 71 08 00 > 0000 (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0a(da2: 00mps0:0: 004: 000): 00Retrying command > 00 00 00 00 00 00 00 28 e6 b9 bc |............(...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:(da3:mps0:0:mps0:0:4:5:0): 0): Retrying command > Retrying command > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Retrying command (per sense data) > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe0001125b20 > (da1:0000 mps0:0: 093: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 58 5c b9 bc |............X\134..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 = |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > SGL for command 0xfffffe00010ed960 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 08 bb b6 bc = |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 = |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 20 84 01 00 00 00 00 00 00 00 00 00 00 00 |.. = .............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 00 ba 71 08 00 > (da1:(da2:mps0:0:4:0): CAM status: SCSI Status Error > mps0:0:(da2:mps0:0:4:0): SCSI status: Check Condition > 3:(da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 0): (da2:Retrying command > mps0:0:(da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > 4:(da3:mps0:0:5:0): CAM status: SCSI Status Error > 0): SGL for command 0xfffffe00010e1f00 > (da3:mps0:0:5:0): SCSI status: Check Condition > Retrying command (per sense data) > 0000 (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 0a(da3: 00mps0:0: 005:(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da > 71 00 00 08 00 > 000): (da0:mps0:0:2:0): CAM status: SCSI Status Error > 00(da0:mps0:0:2:0): SCSI status: Check Condition > Retrying command (per sense data) > 00 00 00 00 00 00 00 40 2f b6 bc |............@/..| > 0010 00(da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power = on, > reset, or bus device reset occurred) > 00(da0: fcmps0:0: 002: 180): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 0a 00 00 |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > SGL for command 0xfffffe000112b460 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 48 9f b9 bc = |............H...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > (da1:mps0:0:3:0): SCSI status: Check Condition > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe000111b710 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:(da2:0000 mps0:0:mps0:0: 093: 000): 4:Retrying command (per sense = data) > 0): 00 00Retrying command > 00 00 00 00 00 00 00 00 4c e1 b8 bc |............L...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 14 3b 7d 05 00 00 00 00 00 00 00 00 00 00 00 = |..;}............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 14 3b 7d 05 00 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:(da0:mps0:0:mps0:0:5:2:0): 0): Error 5, Retries exhausted > Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > (da2:mps0:0:4:0): SCSI status: Check Condition > (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe00011272c0 > (da2:0000 mps0:0: 0b4: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 00 00 00 10 6e b9 bc |.............n..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 c3 5b 7d 00 00 05 00 00 00 00 00 00 00 = |*...[}..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > SGL for command 0xfffffe000110de80 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 00 c3 5b 7d 00 00 05 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0c(da1: 00mps0:0: 003: 000): 00Error 5, Retries exhausted > 00 00 00 00 00 00 00 e0 3e b8 bc |.............>..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 af da 71 00 00 08 00 > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe0001126450 > (da0:0000 mps0:0: 0a2: 000): 00Retrying command (per sense data) > 00(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > 00(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > 00(da0: 00 00 00 00 00 00 3c 63 b9 bc |............ 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00mps0:0: 0a2: 000): = 00Retrying > command > |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 1e e7 e1 05 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da2:mps0:0:4:0): CAM status: Unrecoverable Host Bus Adapter Error > (da2:mps0:0:4:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > SGL for command 0xfffffe00010e4660 > (da3:0000 mps0:0: 0b5: 000): 00Retrying command (per sense data) > 00(da1:mps0:0:3:0): READ(10). CDB: 28 00 1b e2 9e 10 00 00 10 00 > 00(da1:mps0:0:3:0): CAM status: SCSI Status Error > 00(da1:mps0:0:3:0): SCSI status: Check Condition > 00(da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 00(da1: 00mps0:0: 003: 000): 00Retrying command (per sense data) > c8 4c b6 bc |.............L..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. = ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 = |................| > 0040 28 00 00 af 22 10 00 00 10 00 00 00 00 00 00 00 = |(..."...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 12 00 b0 87 88 01 00 00 00 00 10 00 d3 = |................| > 0070 00 e0 41 a9 01 00 00 00 00 00 00 00 00 00 00 00 = |..A.............| > seg0 flags=3D0x12 len=3D0x001000 addr=3D0x000000018887b000 > seg1 flags=3D0xd3 len=3D0x001000 addr=3D0x00000001a941e000 > SGL for command 0xfffffe00010eaa20 > (da1:mps0:0:3:0): READ(10). CDB: 28 00 00 af 22 10 00 00 10 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0b(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 98 97 b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. = ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 02 = |................| > 0040 28 00 1b e2 9c 10 00 00 10 00 00 00 00 00 00 00 = |(...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 10 00 12 00 30 b3 b2 01 00 00 00 00 10 00 d3 = |.....0..........| > 0070 00 d0 b0 b2 01 00 00 00 00 00 00 00 00 00 00 00 = |................| > seg0 flags=3D0x12 len=3D0x001000 addr=3D0x00000001b2b33000 > seg1 flags=3D0xd3 len=3D0x001000 addr=3D0x00000001b2b0d000 > SGL for command 0xfffffe00011211a0 > (da1:mps0:0:3:0): READ(10). CDB: 28 00 1b e2 9c 10 00 00 10 00 > 0000 (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter = Error > 0b(da1: 00mps0:0: 003: 000): 00Retrying command > 00 00 00 00 00 00 00 38 25 b9 bc |............8%..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 f0 5d 20 00 00 05 00 00 00 00 00 00 00 |*...] = ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da2:mps0:0:4:0): WRITE(6). CDB: 0a 1e e7 e1 05 00 > (da2:mps0:0:4:0): CAM status: SCSI Status Error > SGL for command 0xfffffe000110b720 > (da2:mps0:0:4:0): SCSI status: Check Condition > 0000 (da2:mps0:0:4:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 09(da2: 00mps0:0: 004: 000): 00Error 6, Retries exhausted > 00(da2: 00mps0:0: 004: 000): 00Invalidating pack > 00(da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > 00(da0:mps0:0:2:0): CAM status: SCSI Status Error > 58(da0:mps0:0:2:0): SCSI status: Check Condition > 21(da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > b8(da0: bcmps0:0: |2:.0): .Retrying command (per sense data) > ..........X!..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. = ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 1b f2 42 10 00 00 10 00 00 00 00 00 00 00 = |*...B...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 20 00 d7 00 f0 6c 09 00 00 00 00 00 00 00 00 |. = ....l.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x002000 addr=3D0x00000000096cf000 > SGL for command 0xfffffe00010ef100 > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 1b f2 42 10 00 00 10 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter = Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 c0 cc b6 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. = ..| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 1b f2 40 10 00 00 10 00 00 00 00 00 00 00 = |*...@...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 20 00 d7 00 40 8f 0c 00 00 00 00 00 00 00 00 |. = ...@..........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x002000 addr=3D0x000000000c8f4000 > SGL for command 0xfffffe0001122a90 > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 1b f2 40 10 00 00 10 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter = Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 ec 37 b9 bc |.............7..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 41 3d 20 00 00 05 00 00 00 00 00 00 00 |*..A=3D = ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > SGL for command 0xfffffe00010e11e0 > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter = Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 68 25 b6 bc |............h%..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 20 00 00 |............. = ..| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 00 02 10 10 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 20 00 d7 00 70 bf 0c 00 00 00 00 00 00 00 00 |. = ...p..........| > 0070 00 00 a0 88 01 00 00 00 00 00 00 00 00 00 00 00 = |................| > seg0 flags=3D0xd7 len=3D0x002000 addr=3D0x000000000cbf7000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 00 02 10 10 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > SGL for command 0xfffffe0001126ed0 > (da1:mps0:0:3:0): CAM status: SCSI Status Error > 0000 (da1:mps0:0:3:0): SCSI status: Check Condition > 0c(da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 00(da1: 00mps0:0: 003: 000): 00Retrying command (per sense data) > 00 00 00 00 00 00 1c 6b b9 bc |.............k..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 = |................| > 0040 35 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |5...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 00 00 d7 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000000 addr=3D0x0000000000000000 > SGL for command 0xfffffe0001122010 > 0000 0c 00 00 00 00 00 00 00 00 00 00 00 0c 30 b9 bc = |.............0..| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 00 ce 07 e1 00 00 05 00 00 00 00 00 00 00 = |*...............| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > (da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter Error > (da0:mps0:0:(da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 = 05 00 > 2:(da1:mps0:0:3:0): CAM status: SCSI Status Error > 0): (da1:mps0:0:3:0): SCSI status: Check Condition > Retrying command > (da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): WRITE(10). CDB: 2a 00 00 ce 07 e1 00 00 05 00 > (da1:(da0:mps0:0:2:0): CAM status: Unrecoverable Host Bus Adapter = Error > mps0:0:(da0:3:mps0:0:0): 2:Retrying command (per sense data) > 0): Error 5, Retries exhausted > SGL for command 0xfffffe00011209c0 > 0000 0b 00 00 00 00 00 00 00 00 00 00 00 50 1f b9 bc = |............P...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 f0 5d 20 00 00 05 00 00 00 00 00 00 00 |*...] = ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > (da1:mps0:0:3:0): CAM status: Unrecoverable Host Bus Adapter Error > (da1:mps0:0:3:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da0:mps0:0:2:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00 00 = 00 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da0:mps0:0:2:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da0:mps0:0:2:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da0:mps0:0:2:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da3:(da0:mps0:0:mps0:0:5:2:0): 0): Retrying command (per sense data) > Error 6, Retries exhausted > (da1:mps0:0:3:0): WRITE(10). CDB: 2a 00 05 f0 5d 20 00 00 05 00 > (da0:(da1:mps0:0:3:0): CAM status: SCSI Status Error > mps0:0:(da1:mps0:0:3:0): SCSI status: Check Condition > 2:(da1:mps0:0:3:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, > reset, or bus device reset occurred) > 0): (da1:Invalidating pack > mps0:0:3:0): Error 6, Retries exhausted > (da1:mps0:0:3:0): Invalidating pack > SGL for command 0xfffffe000112de60 > 0000 09 00 00 00 00 00 00 00 00 00 00 00 c8 be b9 bc = |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 90 00 00 = |................| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 00 6b 4c 48 00 00 00 00 00 00 00 00 00 00 00 = |..kLH...........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 90 00 d7 00 60 cf 16 00 00 00 00 00 00 00 00 = |.....`..........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x009000 addr=3D0x0000000016cf6000 > SGL for command 0xfffffe0001117420 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 00 6b 4c 48 00 > 0000 (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter = Error > 09(da3: 00mps0:0: 005: 000): 00Retrying command > 00 00 00 00 00 00 00 18 af b8 bc |................| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 50 01 00 = |.............P..| > 0020 00 00 00 00 06 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 0a 00 c5 98 a8 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 50 01 d7 00 f0 3d 84 01 00 00 00 00 00 00 00 = |.P....=3D.........| > 0070 00 00 60 0c 00 00 00 00 00 00 00 00 00 00 00 00 = |..`.............| > seg0 flags=3D0xd7 len=3D0x015000 addr=3D0x00000001843df000 > SGL for command 0xfffffe00010f19b0 > 0000 09 00 00 00 00 00 00 00 00 00 00 00 44 eb b6 bc = |............D...| > 0010 00 00 fc 00 18 00 00 00 00 00 00 00 00 0a 00 00 = |................| > 0020 00 00 00 00 0a 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 = |................| > 0040 2a 00 05 41 3d 20 00 00 05 00 00 00 00 00 00 00 |*..A=3D = ..........| > 0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 = |................| > 0060 00 0a 00 d7 00 c0 70 d0 01 00 00 00 00 00 00 00 = |......p.........| > 0070 00 00 40 13 00 00 00 00 00 00 00 00 00 00 00 00 = |..@.............| > seg0 flags=3D0xd7 len=3D0x000a00 addr=3D0x00000001d070c000 > (da3:mps0:0:5:0): WRITE(6). CDB: 0a 00 c5 98 a8 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da3:mps0:0:5:0): CAM status: Unrecoverable Host Bus Adapter Error > (da3:mps0:0:5:0): Retrying command > (da3:mps0:0:5:0): WRITE(10). CDB: 2a 00 05 41 3d 20 00 00 05 00 > (da3:mps0:0:5:0): CAM status: SCSI Status Error > (da3:mps0:0:5:0): SCSI status: Check Condition > (da3:mps0:0:5:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on, = reset, > or bus device reset occurred) > (da3:mps0:0:5:0): Error 6, Retries exhausted > (da3:mps0:0:5:0): Invalidating pack > pid 6760 (bhyve), uid 0, was killed: out of swap space > pid 1327 (ntpd), uid 0, was killed: out of swap space > ahcich6: Timeout on slot 4 port 0 > ahcich6: is 00000000 cs 00000010 ss 00000000 rs 00000010 tfd 150 serr > 00000000 cmd 0004c417 > (ada6:ahcich6:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 = 40 > 00 00 00 00 00 00 > (ada6:ahcich6:0:0:0): CAM status: Command timeout > (ada6:ahcich6:0:0:0): Error 5, Periph was invalidated > ahcich6: Poll timeout on slot 5 port 0 > ahcich6: is 00000000 cs 00000020 ss 00000000 rs 00000020 tfd 150 serr > 00000000 cmd 0004c517 > (aprobe0:ahcich6:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 = 00 > 00 00 > (aprobe0:ahcich6:0:0:0): CAM status: Command timeout > (aprobe0:ahcich6:0:0:0): Error 5, Retries exhausted >=20 From owner-freebsd-scsi@freebsd.org Fri Dec 9 15:43:31 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F74FC6E594 for ; Fri, 9 Dec 2016 15:43:31 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 410321F2C for ; Fri, 9 Dec 2016 15:43:31 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id uB9FhTf4057069; Fri, 9 Dec 2016 16:43:29 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 66FBFB03; Fri, 9 Dec 2016 16:43:29 +0100 (CET) Message-ID: <584AD121.7040706@omnilan.de> Date: Fri, 09 Dec 2016 16:43:29 +0100 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Kevin Day CC: freebsd-scsi@freebsd.org Subject: Re: sporadic CAM (all devices) outage on 11-stable, mps(4), ahci(4) and bhyve(8) involved. References: <584ABE6B.8060703@omnilan.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Fri, 09 Dec 2016 16:43:29 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 15:43:31 -0000 Bezüglich Kevin Day's Nachricht vom 09.12.2016 16:34 (localtime): > I’m not sure if this is related, but after upgrading from 10.x to 11.0-RELEASE we’re seeing on multiple systems that “camcontrol rescan all” causes all devices to detach in a similar fashion. All mps and achi drives timeout and/or just immediately disappear, then reattach. Interestingly mpr backed drives do not disappear. I'm post r309273, but maybe this solves your problem: https://svnweb.freebsd.org/changeset/base/309273 Best, -harry