From owner-freebsd-bugs@freebsd.org Thu Aug 8 19:41:01 2019 Return-Path: Delivered-To: freebsd-bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 59CDCB08F0 for ; Thu, 8 Aug 2019 19:41:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 464JdT1lDMz4mJl for ; Thu, 8 Aug 2019 19:41:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 39F7DB08EF; Thu, 8 Aug 2019 19:41:01 +0000 (UTC) Delivered-To: bugs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 39B54B08EE for ; Thu, 8 Aug 2019 19:41:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 464JdT0lHDz4mJk for ; Thu, 8 Aug 2019 19:41:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EFA762106 for ; Thu, 8 Aug 2019 19:41:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x78Jf08U020481 for ; Thu, 8 Aug 2019 19:41:00 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x78Jf0pf020480 for bugs@FreeBSD.org; Thu, 8 Aug 2019 19:41:00 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 224496] mpr and mps drivers seems to have issues with large seagate drives Date: Thu, 08 Aug 2019 19:41:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: n@nmc.dev X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Aug 2019 19:41:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224496 --- Comment #12 from n@nmc.dev --- Hi, We are seeing the same issue.=20 Here is more information on our setup : FreeNAS-11.2-U5 FreeBSD 11.2-STABLE amd64 We use 2 x (6x 14TB seagate ironwolf drives ) We also have a 2 TB crucial SSD for L2ARC Issue always comes up after 10-14hours of heavy IO Disk Model : 14 TB Seagate ST14000VN0008 The drives are on 2 different LSI HBAs. Drive that fails are random on both those HBA. Please let us know if you need more information on this, it is impacting our production load. Thank you. Log output for our latest errors : > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 a8 00=20 > 00 00 10 00 00 length 8192 SMID 60 Aborting command 0xfffffe000171f640 > mpr1: Sending reset from mprsas_send_abort for target ID 20 > (da30:mpr1:0:20:0): READ(10). CDB: 28 00 b7 81 49 f0 00 00 08 00 length = 4096 SMID 332 terminated ioc 804b loginfo 31130000 scsi 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(10). CDB: 28 00 b7 81 49 e8 00 00 08 00=20 > length 4096 SMID 703 terminated ioc 804b loginfo 31130000 sc(da30:mpr1:0:= 20:0): READ(10). CDB: 28 00 b7 81 49 f0 00 00 08 00 si 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 77 b8 00=20 > 00 01 00 00 00 length 131072 SMID 510 terminated ioc=20 > 804b(da30:mpr1:0:20:0): CAM status: CCB request completed with an=20 > error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(10). CDB: 28 00 b7 81 49 e8 00 00 08 00=20=20 > loginfo 31130000 scsi 0 state c xfer 0 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 76 b8 00 00 0= 1 00 00 00 length 131072 SMID 938 terminated ioc 804b loginfo 31130000 scsi= 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 75 b8 00 00 0= 1 00 00 00 length 131072 SMID 839 terminated ioc 804b loginfo 31130000 scsi= 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 74 b8 00 00 0= 1 00 00 00 length 131072 SMID 681 terminated ioc 804b loginfo 31130000 scsi= 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 73 b8 00 00 0= 1 00 00 00 length 131072 SMID 647 terminated ioc 804b loginfo 31130000 scsi= 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 72 b8 00 00 0= 1 00 00 00 length 131072 SMID 253 terminated ioc 804b loginfo 31130000 scsi= 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 71 b8 00 00 0= 1 00 00 00 length 131072 SMID 109 terminated ioc 804b loginfo 31130000 scsi= 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 b8 00 00 0= 1 00 00 00 length 131072 SMID 267 terminated ioc 804b loginfo 31130000 scsi= 0 state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 98 00 00 0= 0 10 00 00 length 8192 SMID 506 terminated ioc 804b loginfo 31130000 scsi 0= state c xfer 0 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 88 00 00 0= 0 10 00 00 length 8192 SMID 774 terminated ioc 804b loginfo 31130000 scsi 0= state c xfer 0 > (da30:mpr1:0:20:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00=20 > 00 00 00 length 0 SMID 281 terminated ioc 804b loginfo 31140000 scsi 0=20 > state c xfer 0 > mpr1: Unfreezing devq for target ID 20 > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 77 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 76 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 75 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 74 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 73 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 72 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 71 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 b8 00 00=20 > 01 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 a8 00 00=20 > 00 10 00 00 > (da30:mpr1:0:20:0): CAM status: Command timeout > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 98 00 00=20 > 00 10 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(16). CDB: 88 00 00 00 00 01 05 1a 70 88 00 00=20 > 00 10 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00=20 > 00 00 00 > (da30:mpr1:0:20:0): CAM status: CCB request completed with an error > (da30:mpr1:0:20:0): Retrying command > (da30:mpr1:0:20:0): READ(10). CDB: 28 00 b7 81 49 f0 00 00 08 00 > (da30:mpr1:0:20:0): CAM status: SCSI Status Error > (da30:mpr1:0:20:0): SCSI status: Check Condition > (da30:mpr1:0:20:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on,=20 > reset, or bus device reset occurred) > (da30:mpr1:0:20:0): Retrying command (per sense data) > ctl_datamove: tag 0x855ffd44 on (2:3:106) aborted > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 68 00 00 10 00=20 > length 8192 SMID 486 Aborting command 0xfffffe0001745aa0 > mpr1: Sending reset from mprsas_send_abort for target ID 22 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 c1 ae 62 38 00 00 08 00 length = 4096 SMID 105 terminated ioc 804b loginfo 31130000 scsi 0 state c xfer 0 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 6c 78 00 00 a0 00=20 > length 81920 SMID 467 terminated ioc 804b loginfo 31130000 s(da32:mpr1:0:= 22:0): READ(10). CDB: 28 00 c1 ae 62 38 00 00 08 00 csi 0 state c xfer 0 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 6b 78 00 01 00 00=20 > length 131072 SMID 959 terminated ioc 804b loginfo 31130000 (da32:mpr1:0:= 22:0): CAM status: CCB request completed with an error scsi 0 state c xfer 0 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 6a 78 00 01 00 00=20 > length 131072 SMID 346 terminated ioc 804b loginfo 31130000 (da32:scsi=20 > 0 state c xfer 0 > mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 69 78 00 01 00 00=20 > length 131072 SMID 627 terminated ioc 804b loginfo 31130000 scsi 0=20 > state c xfer 0 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 6c 78 00 00 a0 00=20 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 68 78 00 01 00 00=20 > length 131072 SMID 455 terminated ioc 804b loginfo 31130000 (da32:mpr1:0:= 22:0): CAM status: CCB request completed with an error scsi 0 state c xfer 0 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 67 78 00 01 00 00=20 > length 131072 SMID 951 terminated ioc 804b loginfo 31130000 (da32:scsi=20 > 0 state c xfer 0 > mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 66 78 00 01 00 00=20 > length 131072 SMID 822 terminated ioc 804b loginfo 31130000 (da32:mpr1:0:= 22:0): READ(10). CDB: 28 00 ca 8b 6b 78 00 01 00 00 scsi 0 state c xfer 0 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 78 00 01 00 00=20 > length 131072 SMID 155 terminated ioc 804b loginfo 31130000 (da32:mpr1:0:= 22:0): CAM status: CCB request completed with an error scsi 0 state c xfer 0 > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 58 00 00 10 00=20 > length 8192 SMID 495 terminated ioc 804b loginfo 31130000 sc(da32:si 0=20 > state c xfer 0 > mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 48 00 00 10 00=20 > length 8192 SMID 494 terminated ioc 804b loginfo 31130000 sc(da32:mpr1:0:= 22:0): READ(10). CDB: 28 00 ca 8b 6a 78 00 01 00 00 si 0 state c xfer 0 > (da32:mpr1:0:22:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00=20 > 00 00 00 length 0 SMID 726 terminated ioc 804b loginfo=20 > 3(da32:mpr1:0:22:0): CAM status: CCB request completed with an error > 1140000 scsi 0 state c xfer 0 > mpr1: Unfreezing devq for target ID 22 > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 69 78 00 01 00 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 68 78 00 01 00 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 67 78 00 01 00 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 66 78 00 01 00 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 78 00 01 00 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 68 00 00 10 00 > (da32:mpr1:0:22:0): CAM status: Command timeout > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 58 00 00 10 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 ca 8b 65 48 00 00 10 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): SYNCHRONIZE CACHE(10). CDB: 35 00 00 00 00 00 00=20 > 00 00 00 > (da32:mpr1:0:22:0): CAM status: CCB request completed with an error > (da32:mpr1:0:22:0): Retrying command > (da32:mpr1:0:22:0): READ(10). CDB: 28 00 c1 ae 62 38 00 00 08 00 > (da32:mpr1:0:22:0): CAM status: SCSI Status Error > (da32:mpr1:0:22:0): SCSI status: Check Condition > (da32:mpr1:0:22:0): SCSI sense: UNIT ATTENTION asc:29,0 (Power on,=20 > reset, or bus device reset occurred) > (da32:mpr1:0:22:0): Retrying command (per sense data) --=20 You are receiving this mail because: You are the assignee for the bug.=