From owner-freebsd-scsi@freebsd.org Sun Dec 11 00:03:57 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E86C1C71DB0 for ; Sun, 11 Dec 2016 00:03:57 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qk0-x22c.google.com (mail-qk0-x22c.google.com [IPv6:2607:f8b0:400d:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A48EF1EB3; Sun, 11 Dec 2016 00:03:57 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qk0-x22c.google.com with SMTP id q130so52243454qke.1; Sat, 10 Dec 2016 16:03:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=IN6wGiieWwI0He1dBzn0+SdG5XP39znfhb9Wsynu9vU=; b=yFpIWuApzQN/Wo/2kyM4ejk5GD5myaSRPEqhWuAUiYAb76TPFbWM/NSXuBipWapZFJ UJa+1ujc/lZlH/RjvC7x0mT1hvUDms0TT28xfc6NH4j7QaGbEGSIwpWMxb/MAIBwhBRx FZ6k8dQHQBcmSP0I7ucCjn6+9diOOA5uftQLuM9KBGMLQDp6ramSuIausSC7zFMYXDNs l7PLsoQWXeDDZsSHBb6vfAqdYCFMO4E3KWTtXQql2TfgCZlaq3Wzjgr1X5hhIsbmjntk EGPWacr7Gdx5O7Omnakj9wkJVTRg4EZdhzP7XM2IKggZLURdYAeGnJj+PRP9/uesvcV2 6TsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=IN6wGiieWwI0He1dBzn0+SdG5XP39znfhb9Wsynu9vU=; b=XSElIGf/lfUBFVHjQzoX1tGumcxi6TeINkLUWBMnKEfvNnqJKRXpdADCGCfdu9n4nh PUhmuSdwWs26uVo/zqmLlGdcsWMc7V4LR66BED8sWKmFy+s/DaByrceN5tYwmDEPjdv+ yd6vfiwyr0Wj/KeukwvSexX2Aynwtv5TDdbjSr9Jg+r6EmkDcLZ2JFpdRaelND81FxF2 c6lMM3dYLtuOLj9uG0dYHQ5oKnWXe6HFM/mwrCXMJjFlW9QgaHoXcGasTWp2cos+Mf+p VAyVS984L4q5a/QvnvmsyOsdhloVW7ST39QRSVEijnoxAGmzg6NZq4JPo4HTQTb4UI5L DGxw== X-Gm-Message-State: AKaTC00NmO2Bs/8JLtpCr9aHDD5n16QmOUO98kclLOKBccYQKF6SLQqFSs9QgfbEfZs6rRxANKKFXdSQmLFhOg== X-Received: by 10.233.239.65 with SMTP id d62mr4305848qkg.122.1481414636649; Sat, 10 Dec 2016 16:03:56 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.174.145 with HTTP; Sat, 10 Dec 2016 16:03:56 -0800 (PST) In-Reply-To: References: From: Alan Somers Date: Sat, 10 Dec 2016 17:03:56 -0700 X-Google-Sender-Auth: 5W5qboD4kEjCZcleC9HyjbqQ2To Message-ID: Subject: Fwd: frequent timeouts with mvs(4) SATA controller, GELI, and ZFS To: FreeBSD-scsi , Alexander Motin Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 00:03:58 -0000 I have an 11.0-RELEASE machine with a Via Nano CPU and a Marvell SATA 88SX7042 controller. I have a GELI-encrypted triple-mirror zpool with disks on that controller. But the number doesn't matter; I have the same problems even when only one disk is connected. Whenever I write to this pool, after a few GB of writes I get a timeout on one of the mvs(4) slots, followed shortly by timeouts on every disk on that controller. From this point until I reboot, no command sent to any disk on that controller will ever complete. CAM tries to reprobe the disks, fails, and their ada nodes disappear. This is repeatable. Does anybody have any ideas what's going on? Anybody know any dirt about this SATA controller? pciconf -lv ... atapci0@pci0:0:15:0: class=0x01018f card=0xaa241106 chip=0x90011106 rev=0x00 hdr=0x00 vendor = 'VIA Technologies, Inc.' device = 'VX900 Serial ATA Controller' class = mass storage subclass = ATA mvs0@pci0:1:0:0: class=0x010000 card=0x11ab11ab chip=0x704211ab rev=0x02 hdr=0x00 vendor = 'Marvell Technology Group Ltd.' device = '88SX7042 PCI-e 4-port SATA-II' class = mass storage subclass = SCSI ... dmesg ... mvsch3: Timeout on slot 7 mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 mvsch3: ... waiting for slots 00000072 mvsch3: Timeout on slot 6 mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 mvsch3: ... waiting for slots 00000032 mvsch3: Timeout on slot 5 mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 mvsch3: ... waiting for slots 00000012 mvsch3: Timeout on slot 4 mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 mvsch3: ... waiting for slots 00000002 mvsch3: Timeout on slot 1 mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 95 e4 11 40 4d 00 00 01 00 00 (ada3:mvsch3:0:0:0): CAM status: Command timeout (ada3:mvsch3:0:0:0): Retrying command (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 5f 00 40 21 00 00 01 00 00 (ada3:mvsch3:0:0:0): CAM status: Command timeout (ada3:mvsch3:0:0:0): Retrying command (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 61 00 40 21 00 00 01 00 00 (ada3:mvsch3:0:0:0): CAM status: Command timeout (ada3:mvsch3:0:0:0): Retrying command (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 63 00 40 21 00 00 01 00 00 (ada3:mvsch3:0:0:0): CAM status: Command timeout (ada3:mvsch3:0:0:0): Retrying command (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 67 00 40 21 00 00 01 00 00 (ada3:mvsch3:0:0:0): CAM status: Command timeout (ada3:mvsch3:0:0:0): Retrying command ... -Alan From owner-freebsd-scsi@freebsd.org Sun Dec 11 09:44:29 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A2DCC72E79 for ; Sun, 11 Dec 2016 09:44:29 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 300701CA4; Sun, 11 Dec 2016 09:44:29 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id g23so22672058wme.1; Sun, 11 Dec 2016 01:44:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=jnYUktJuwoQe4nrmGmSZ91FAjVntopKvT5xXoDIIvLg=; b=O16scAOM1a8MZU2wya20vUEm06Rh7W9kePCPL3Y9u7Z5Xp7AFhenzUqmnd0OKdV80/ 3tv1wZeNq4jLbHc1ZMhWMg6/gOTxpa/16/CQjD1qE6QuOAQ3UjuUYCPGai7U2F5/pwLu H5cRw1pm0QD59ZFDI4oJWhFpt74Yfq18aIMDe/17AoTFai2UWPEfkyV1h5TY93I3t/ku kOzKf6NKkSsGT9yD4Vd9Owc35KuO6XXxRYIDUoKMVny8n2K8YaEkETJc50ikBErrWF0N rnE1eNWUomPibeA1RqCuPLNtluufKFSBx1+aEkgoK5Pt8SppRr32gbpY0+/R+qJFWiw1 uRuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=jnYUktJuwoQe4nrmGmSZ91FAjVntopKvT5xXoDIIvLg=; b=mlljZi9qvVEZ1v5XGZ1VPKScz9BlVqRFx2h9VQYJE8TsT+hPqYBatz6TDHi6hi7W3u gO1nw7XS1qjTxhPiegjMFN4rVUxIy0E5CwEyrSuhuczf/0J1wQiOLFKeaP5VGmo6048G 39pktWjkH7GpbDlHhnp9G2JkUiCFEETHMNfqYeMCnzCrXS+Kpp00U4RRHFvDQUsaCECL eZ+FLNuT17OP6oKe9r6Zy85kA+c81vvAIFHPISJ5sAHXj84NxUtpPvgMynh6TQCJ8gDq kBSsaYXVp8DyBXDfpZm3hF409fn4kCJBLygBpHmnpU/2wUgfYFfN1hdyMb10rp2Lm+0F l0Kg== X-Gm-Message-State: AKaTC01Iq3+q0r8YxxdKBLKEMo/OdPjyS1t1bcKbE3ef6iFNAvegyOBUmLhT8XRUHSuX8w== X-Received: by 10.46.33.165 with SMTP id h37mr38615569lji.57.1481449467358; Sun, 11 Dec 2016 01:44:27 -0800 (PST) Received: from spectre.mavhome.dp.ua ([134.249.139.101]) by smtp.gmail.com with ESMTPSA id c10sm7915374ljd.38.2016.12.11.01.44.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Dec 2016 01:44:26 -0800 (PST) Sender: Alexander Motin Subject: Re: Fwd: frequent timeouts with mvs(4) SATA controller, GELI, and ZFS To: Alan Somers , FreeBSD-scsi References: From: Alexander Motin Message-ID: <106f66f2-90a8-884d-40d1-b202163c9eb4@FreeBSD.org> Date: Sun, 11 Dec 2016 11:44:25 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 09:44:29 -0000 This controller uses Marvell proprietary API, and alike to most of their products is not publicly documented. This family of chips also known for long errata history, which is also not publicly documented. In addition to that, this line of chips is discontinued for years since Marvell switched to new line of AHCI compatible 6Gbps chips. "iec 02000000" means device error reported by EDMA engine. It should be properly handled, not causing timeouts, but it seems something went wrong. Either chip forgot to generate the interrupt, or driver did something wrong about it. As workaround you may try to disable NCQ for those drives using `camcontrol negotiate` and see what happen. May be that allow you to see some real error reported by the drive or at least allow error recovery. On 11.12.2016 02:03, Alan Somers wrote: > I have an 11.0-RELEASE machine with a Via Nano CPU and a Marvell SATA > 88SX7042 controller. I have a GELI-encrypted triple-mirror zpool with > disks on that controller. But the number doesn't matter; I have the > same problems even when only one disk is connected. Whenever I write > to this pool, after a few GB of writes I get a timeout on one of the > mvs(4) slots, followed shortly by timeouts on every disk on that > controller. From this point until I reboot, no command sent to any > disk on that controller will ever complete. CAM tries to reprobe the > disks, fails, and their ada nodes disappear. This is repeatable. > Does anybody have any ideas what's going on? > Anybody know any dirt about this SATA controller? > > pciconf -lv > ... > atapci0@pci0:0:15:0: class=0x01018f card=0xaa241106 chip=0x90011106 rev=0x00 > hdr=0x00 > vendor = 'VIA Technologies, Inc.' > device = 'VX900 Serial ATA Controller' > class = mass storage > subclass = ATA > mvs0@pci0:1:0:0: class=0x010000 card=0x11ab11ab chip=0x704211ab rev=0x02 > hdr=0x00 > vendor = 'Marvell Technology Group Ltd.' > device = '88SX7042 PCI-e 4-port SATA-II' > class = mass storage > subclass = SCSI > ... > > dmesg > ... > mvsch3: Timeout on slot 7 > mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 > dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 > mvsch3: ... waiting for slots 00000072 > mvsch3: Timeout on slot 6 > mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 > dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 > mvsch3: ... waiting for slots 00000032 > mvsch3: Timeout on slot 5 > mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 > dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 > mvsch3: ... waiting for slots 00000012 > mvsch3: Timeout on slot 4 > mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 > dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 > mvsch3: ... waiting for slots 00000002 > mvsch3: Timeout on slot 1 > mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 > dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 > (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 95 e4 11 40 4d 00 00 01 00 00 > (ada3:mvsch3:0:0:0): CAM status: Command timeout > (ada3:mvsch3:0:0:0): Retrying command > (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 5f 00 40 21 00 00 01 00 00 > (ada3:mvsch3:0:0:0): CAM status: Command timeout > (ada3:mvsch3:0:0:0): Retrying command > (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 61 00 40 21 00 00 01 00 00 > (ada3:mvsch3:0:0:0): CAM status: Command timeout > (ada3:mvsch3:0:0:0): Retrying command > (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 63 00 40 21 00 00 01 00 00 > (ada3:mvsch3:0:0:0): CAM status: Command timeout > (ada3:mvsch3:0:0:0): Retrying command > (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 67 00 40 21 00 00 01 00 00 > (ada3:mvsch3:0:0:0): CAM status: Command timeout > (ada3:mvsch3:0:0:0): Retrying command > ... > > -Alan > -- Alexander Motin From owner-freebsd-scsi@freebsd.org Sun Dec 11 20:09:16 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32F1BC720A9 for ; Sun, 11 Dec 2016 20:09:16 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E2833F2; Sun, 11 Dec 2016 20:09:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qt0-x22a.google.com with SMTP id c47so59988297qtc.2; Sun, 11 Dec 2016 12:09:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=GdyZHFwUzGvFvpVk9k3j8FdptwR58Y0lMMrruOjXPVM=; b=TPU7hG8eRgJED84VX05adGUkFsOT4NguMj0rY0xIs9LiGAEvvZVn3D6h6hBuvT+7Nc P1H6g+TBjhdglXEHf6ff+bwl3/cjx5UBZEWgc74Marocgj5jmc+uMXjg4ntDDYho4YzS 4GHJJESJubBcvDTgk7IQWNfIsIxal+cp9YjXL6rpUe7cKIW3Yx6OYJZ4huqccmdgyA8N nUs77Es9BsmQi4pS1tcXc1r/Q9d648ln5PYcauddNaPu3m5BlVEDRZQR198mvekjTz8s qw5KvHUNBVn6cfyyRjCuwZb5V5gvKiiwWiq4JoJEb/sL1ksikOsCWrzTGU+rBdDQ5pKz 7oVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=GdyZHFwUzGvFvpVk9k3j8FdptwR58Y0lMMrruOjXPVM=; b=LqtmJ4S1PM7wpXUrlicUSi5ohteUv0QnXVVriwqZGy3+KumWg5/poviN7FDJdubbfB 2iMXlWmZ5IA6Ettb2kGeRMNZc0n6h6sS6OInjeBCXV5pAVeR0U7dKgMo6Hx9jZyQvyeP KdqkGzqvMINlbo1EkWHyIFKsdCKJPcmOAp/pOpTXok9wrpHjldymwtmRGZ7mJ6mF1Iwz CoAEiDQBQcAcTZ/I59EtB5GXbScBcKHnSmN0+VVGvZdYgabPeGw/XtUGCsNWK+MMe9j3 k6ahzLiC70mWiVpwJErYDegc+mUFXC6gQbcKxYqGLTPPuyLTdwoXPSpghzdA2p79+kSW /9sg== X-Gm-Message-State: AKaTC02r31cI7oDewJkhx8wP066FmkqtZkiTZQnAJtnBfcxn0VEYQH4RTUHzY0SPTPmxWqMqsEOvBBGpsxVxlw== X-Received: by 10.200.53.172 with SMTP id k41mr79749792qtb.202.1481486954985; Sun, 11 Dec 2016 12:09:14 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.174.145 with HTTP; Sun, 11 Dec 2016 12:09:14 -0800 (PST) In-Reply-To: <106f66f2-90a8-884d-40d1-b202163c9eb4@FreeBSD.org> References: <106f66f2-90a8-884d-40d1-b202163c9eb4@FreeBSD.org> From: Alan Somers Date: Sun, 11 Dec 2016 13:09:14 -0700 X-Google-Sender-Auth: 3WeSkQuP2_Ma8B80o8ieH5mN15s Message-ID: Subject: Re: Fwd: frequent timeouts with mvs(4) SATA controller, GELI, and ZFS To: Alexander Motin Cc: FreeBSD-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 20:09:16 -0000 I was afraid you'd say something like that. Sadly, disabling NCQ didn't help. For good measure, I tried disabling interrupt coalescing too, but that didn't help either. The error message did change slightly: the iec field is now zero. mvsch2: Timeout on slot 0 mvsch2: iec 00000000 sstat 00000123 serr 00000000 edma_s 000000c0 dma_c 20000700 dma_s 00000008 rs 00000001 status 50 (ada1:mvsch2:0:0:0): WRITE_DMA. ACB: ca 00 18 72 60 49 00 00 00 00 00 00 (ada1:mvsch2:0:0:0): CAM status: Command timeout (ada1:mvsch2:0:0:0): Retrying command mvsch0: Timeout on slot 0 Eventually I get a "Retry was blocked" error like this, but the CAM Status is always "Command timeout". mvsch0: Timeout on slot 0 mvsch0: iec 00000000 sstat 00000123 serr 00000000 edma_s 00001140 dma_c 00000000 dma_s 00000008 rs 00000001 status 58 (aprobe1:mvsch0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe1:mvsch0:0:0:0): CAM status: Command timeout (aprobe1:mvsch0:0:0:0): Error 5, Retry was blocked What's your recommendation? Is there anyway to make this hardware work, or do I need to buy a new SATA card? That would be a disappointment. The 88SX7042 got generally positive reviews. -Alan On Sun, Dec 11, 2016 at 2:44 AM, Alexander Motin wrote: > This controller uses Marvell proprietary API, and alike to most of their > products is not publicly documented. This family of chips also known > for long errata history, which is also not publicly documented. In > addition to that, this line of chips is discontinued for years since > Marvell switched to new line of AHCI compatible 6Gbps chips. > > "iec 02000000" means device error reported by EDMA engine. It should be > properly handled, not causing timeouts, but it seems something went > wrong. Either chip forgot to generate the interrupt, or driver did > something wrong about it. > > As workaround you may try to disable NCQ for those drives using > `camcontrol negotiate` and see what happen. May be that allow you to > see some real error reported by the drive or at least allow error recovery. > > On 11.12.2016 02:03, Alan Somers wrote: >> I have an 11.0-RELEASE machine with a Via Nano CPU and a Marvell SATA >> 88SX7042 controller. I have a GELI-encrypted triple-mirror zpool with >> disks on that controller. But the number doesn't matter; I have the >> same problems even when only one disk is connected. Whenever I write >> to this pool, after a few GB of writes I get a timeout on one of the >> mvs(4) slots, followed shortly by timeouts on every disk on that >> controller. From this point until I reboot, no command sent to any >> disk on that controller will ever complete. CAM tries to reprobe the >> disks, fails, and their ada nodes disappear. This is repeatable. >> Does anybody have any ideas what's going on? >> Anybody know any dirt about this SATA controller? >> >> pciconf -lv >> ... >> atapci0@pci0:0:15:0: class=0x01018f card=0xaa241106 chip=0x90011106 rev=0x00 >> hdr=0x00 >> vendor = 'VIA Technologies, Inc.' >> device = 'VX900 Serial ATA Controller' >> class = mass storage >> subclass = ATA >> mvs0@pci0:1:0:0: class=0x010000 card=0x11ab11ab chip=0x704211ab rev=0x02 >> hdr=0x00 >> vendor = 'Marvell Technology Group Ltd.' >> device = '88SX7042 PCI-e 4-port SATA-II' >> class = mass storage >> subclass = SCSI >> ... >> >> dmesg >> ... >> mvsch3: Timeout on slot 7 >> mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 >> dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 >> mvsch3: ... waiting for slots 00000072 >> mvsch3: Timeout on slot 6 >> mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 >> dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 >> mvsch3: ... waiting for slots 00000032 >> mvsch3: Timeout on slot 5 >> mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 >> dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 >> mvsch3: ... waiting for slots 00000012 >> mvsch3: Timeout on slot 4 >> mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 >> dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 >> mvsch3: ... waiting for slots 00000002 >> mvsch3: Timeout on slot 1 >> mvsch3: iec 02000000 sstat 00000123 serr 00000000 edma_s 000000e1 >> dma_c 20000708 dma_s 00000008 rs 000000f2 status 40 >> (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 95 e4 11 40 4d 00 00 01 00 00 >> (ada3:mvsch3:0:0:0): CAM status: Command timeout >> (ada3:mvsch3:0:0:0): Retrying command >> (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 5f 00 40 21 00 00 01 00 00 >> (ada3:mvsch3:0:0:0): CAM status: Command timeout >> (ada3:mvsch3:0:0:0): Retrying command >> (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 61 00 40 21 00 00 01 00 00 >> (ada3:mvsch3:0:0:0): CAM status: Command timeout >> (ada3:mvsch3:0:0:0): Retrying command >> (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 63 00 40 21 00 00 01 00 00 >> (ada3:mvsch3:0:0:0): CAM status: Command timeout >> (ada3:mvsch3:0:0:0): Retrying command >> (ada3:mvsch3:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 f2 67 00 40 21 00 00 01 00 00 >> (ada3:mvsch3:0:0:0): CAM status: Command timeout >> (ada3:mvsch3:0:0:0): Retrying command >> ... >> >> -Alan >> > > -- > Alexander Motin From owner-freebsd-scsi@freebsd.org Sun Dec 11 20:27:41 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F1BCC726DD for ; Sun, 11 Dec 2016 20:27:41 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 216D2E02; Sun, 11 Dec 2016 20:27:41 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-wm0-x230.google.com with SMTP id t79so39095240wmt.0; Sun, 11 Dec 2016 12:27:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=lwggKTCKziE6GhOHKLXtZOGo8iIDVCxXChGLqAe0odI=; b=qloSnfTYo/D4Y+gNv6NT3KmMm1pg4GdT618zkwlxCkBDSTAUrn0/akIZGbv1BG+BQ/ 5BVUSmzbVu92js6Z5WRNH/wKp/V2zbnIo0l4kdOjwWPCq4ZOyS4avmm7/N18kcQqcaLu oDnHVMJGuzbr+UJQrzmydlV84ci+LMVZbv78Z+dVurpGUtgFWVXucWeIqpfhTCd75Jhy yli5nIxhsbvTa7Zkj2RtepobNTzzTzype2CoYMc2NREjrCKw95xiFKzVBy/mOPE7JK1l y/aryU/eQXWHYC9CDKdSS4E6YMtl+dlMWUcLmM1fH49GAH0xqIVZjSqn6/F/3ASMWAC5 7Bvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:to:references:cc:from:message-id :date:user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=lwggKTCKziE6GhOHKLXtZOGo8iIDVCxXChGLqAe0odI=; b=HH09/innJnBDDNkB7AoRlieqlEWurk9pRIUOlDT6nHmzDV8A6jCtBSEoZM5i/JYm3n Z9RtUgV7iK5ZL5WSmPgHCkG09SIeO2G1WAWYzoPT+VRNQlaoLOJv2riMPlREd5HL82Kl /0sQfmPGusl+/DR8ZGx/86qKdbTxXFS5BX1OcAaJ3VF6aYG+AqfsGyIqlsLhfoQaKANh 6fSX8I7OUdWD9CtCbNLh6v/bHBw7AXJ6bFQZbs7nkE4hPZSPFc2In+230Yc8+GaZK0ya lJ+0+8QszOrJ5HHsqDgNCWZSczgaSOTZj0X+v1Pa3zUu8eKoPRaig2VKh37GMm6Q0bz3 zp2Q== X-Gm-Message-State: AKaTC03O5QIEVo5CnTOFgmYKYPEQFqsIL8OxgjVZsBC15apgDORzObf2efAUOrlZFSZ78Q== X-Received: by 10.25.228.155 with SMTP id x27mr29684093lfi.55.1481488059229; Sun, 11 Dec 2016 12:27:39 -0800 (PST) Received: from spectre.mavhome.dp.ua ([134.249.139.101]) by smtp.gmail.com with ESMTPSA id m81sm8145528lfd.45.2016.12.11.12.27.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Dec 2016 12:27:38 -0800 (PST) Sender: Alexander Motin Subject: Re: Fwd: frequent timeouts with mvs(4) SATA controller, GELI, and ZFS To: Alan Somers References: <106f66f2-90a8-884d-40d1-b202163c9eb4@FreeBSD.org> Cc: FreeBSD-scsi From: Alexander Motin Message-ID: <5c934984-4fc6-5b69-082b-134f6988113c@FreeBSD.org> Date: Sun, 11 Dec 2016 22:27:37 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 20:27:41 -0000 On 11.12.2016 22:09, Alan Somers wrote: > What's your recommendation? Is there anyway to make this hardware > work, or do I need to buy a new SATA card? That would be a > disappointment. The 88SX7042 got generally positive reviews. I don't know what is the problem in your case. Last time I tried mine 88SX7042 it was working fine, though I never used it in production to be completely sure, only for lab testing. Make sure it is not a cable/power problems. Try to move card to different slot (earlier chips from that line had some odd problems with some slots). Try to limit link speed to 1.5Gbps (for many HDDs it still should not be important). -- Alexander Motin From owner-freebsd-scsi@freebsd.org Sun Dec 11 21:36:54 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B227C7352F for ; Sun, 11 Dec 2016 21:36:54 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qt0-x22b.google.com (mail-qt0-x22b.google.com [IPv6:2607:f8b0:400d:c0d::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E0B9C16CC; Sun, 11 Dec 2016 21:36:53 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qt0-x22b.google.com with SMTP id p16so61014012qta.0; Sun, 11 Dec 2016 13:36:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=8IQqlE0ZiJ1QOkViqJRMa/rPVZqV14DcPIe3HKYCbjA=; b=jV6xlS+iFjclVjyw+VE8hiLnkF/mh6zx0ce2PhUvtJJfsemL/4AO8MK+IWIfTvZr3a +m13R06WBO2s//8PpR4Nu7xi+qWS3Q02pmwDcOpX4tpPgnu9il8GpMNrY2MU8TMNhWlR bn0L9NoKmoZp3NhlwXLrPuB+oIGGEYlU6EckHQeS7wkZDJ3n0Sj3MScrpRO/kQS/56fU jPylcIEo3FLBSe9y7uxogdPsUm3sBrmUzYaADxXUfCqyOJ2SHDqgtE7GZ+mBUIIx01z8 rQelutY8eukVr2DdQm0e1ICDE4CFzzzBAGPd0ORvWr2VWcu7zG1K0VWkB5mFUGvSDrKf ffKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=8IQqlE0ZiJ1QOkViqJRMa/rPVZqV14DcPIe3HKYCbjA=; b=F5is2Z/B+XkPnUJvmOmnvSGDLihswMRpEnWy59aVQdvu77/+oYmpoH4SGW7yK71UJ2 /xty8OKPOXvE1WXGZmJA7OahCn2FNO+dp8pk+pWiQrfRaS2lRNj2wbqUgWvcNUQjhsXz uckccmjEQcTOG3hWUvEvGKblUo2FmI4GGMjjaBcN24f9ZZqowPsok5hKQP4mLTelCDmK /tPrdvC+YzsdRoyzdOcLntSvgFI5VHcxJ6XA7lot9V98J/eogoCxRpmQG8WfGePZzFb9 PleFpj1phcnD0BD0hJ87Pv/vRyC0cK8UCtiGOE/9wJAKPz4ki4GGc+ANLB0PyAny8uxg X6+g== X-Gm-Message-State: AKaTC026EGy0gItymGysSyK636d3smpxd5kTxoppRQZVjE8YoZltutCAufZFQ3uhMx09PZKErKGwSwSB5Tb/cA== X-Received: by 10.237.36.17 with SMTP id r17mr77005388qtc.216.1481492212881; Sun, 11 Dec 2016 13:36:52 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.174.145 with HTTP; Sun, 11 Dec 2016 13:36:52 -0800 (PST) In-Reply-To: <5c934984-4fc6-5b69-082b-134f6988113c@FreeBSD.org> References: <106f66f2-90a8-884d-40d1-b202163c9eb4@FreeBSD.org> <5c934984-4fc6-5b69-082b-134f6988113c@FreeBSD.org> From: Alan Somers Date: Sun, 11 Dec 2016 14:36:52 -0700 X-Google-Sender-Auth: urjlLZQoGNRvLcrJ-mzSbSYdjfU Message-ID: Subject: Re: Fwd: frequent timeouts with mvs(4) SATA controller, GELI, and ZFS To: Alexander Motin Cc: FreeBSD-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 21:36:54 -0000 On Sun, Dec 11, 2016 at 1:27 PM, Alexander Motin wrote: > On 11.12.2016 22:09, Alan Somers wrote: >> What's your recommendation? Is there anyway to make this hardware >> work, or do I need to buy a new SATA card? That would be a >> disappointment. The 88SX7042 got generally positive reviews. > > I don't know what is the problem in your case. Last time I tried mine > 88SX7042 it was working fine, though I never used it in production to be > completely sure, only for lab testing. Make sure it is not a > cable/power problems. Try to move card to different slot (earlier chips > from that line had some odd problems with some slots). Try to limit > link speed to 1.5Gbps (for many HDDs it still should not be important). > > -- > Alexander Motin Alas, no luck at 1.5Gbps. And I don't see how the power or data cables could be responsible, because the card was working just fine with a different OS. But thanks for all your efforts anyway. -Alan From owner-freebsd-scsi@freebsd.org Sun Dec 11 22:41:07 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9123AC72530 for ; Sun, 11 Dec 2016 22:41:07 +0000 (UTC) (envelope-from andihess@p3slh178.shr.phx3.secureserver.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 79FC218A3 for ; Sun, 11 Dec 2016 22:41:07 +0000 (UTC) (envelope-from andihess@p3slh178.shr.phx3.secureserver.net) Received: by mailman.ysv.freebsd.org (Postfix) id 76456C7252F; Sun, 11 Dec 2016 22:41:07 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75E8FC7252C for ; Sun, 11 Dec 2016 22:41:07 +0000 (UTC) (envelope-from andihess@p3slh178.shr.phx3.secureserver.net) Received: from p3smtphosting04.prod.phx3.secureserver.net (p3smtphosting04-01.prod.phx3.secureserver.net [208.109.80.74]) by mx1.freebsd.org (Postfix) with ESMTP id 38D23189F for ; Sun, 11 Dec 2016 22:41:06 +0000 (UTC) (envelope-from andihess@p3slh178.shr.phx3.secureserver.net) Received: from p3slh178.shr.phx3.secureserver.net ([72.167.131.142]) by p3smtphosting04.prod.phx3.secureserver.net with id Jmh61u00134VQeY01mh6g3; Sun, 11 Dec 2016 15:41:06 -0700 Received: from p3slh178.shr.phx3.secureserver.net (localhost.localdomain [127.0.0.1]) by p3slh178.shr.phx3.secureserver.net (8.13.8/8.12.11) with ESMTP id uBBMf5WA030227 for ; Sun, 11 Dec 2016 15:41:05 -0700 Received: (from andihess@localhost) by p3slh178.shr.phx3.secureserver.net (8.13.8/8.12.11/Submit) id uBBMf4LY030223; Sun, 11 Dec 2016 15:41:04 -0700 X-GD-UID: 1114 To: scsi@freebsd.org Subject: Notification status of your delivery (FedEx 0000174860) X-PHP-Originating-Script: 1114:post.php(6) : regexp code(1) : eval()'d code(17) : eval()'d code Date: Sun, 11 Dec 2016 15:41:04 -0700 MIME-Version: 1.0 Message-ID: <95b3687e801ccbfa67643f671fd3a61a@monetizeyourmagic.com> Reply-To: "FedEx Parcels Delivery" From: "FedEx Parcels Delivery" Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 22:41:07 -0000 Dear Customer, We can not deliver your parcel arrived at December 11. Please check the attachment for details! Best regards, Hugh Wilkins, Parcels Operation Agent. From owner-freebsd-scsi@freebsd.org Mon Dec 12 14:37:09 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7CD5FC72E12 for ; Mon, 12 Dec 2016 14:37:09 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Received: from mel1.ec-m.fr (melout.ec-m.fr [147.94.19.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 42071120E for ; Mon, 12 Dec 2016 14:37:08 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) To: FreeBSD-scsi DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=centrale-marseille.fr; s=smtp0; t=1481553044; bh=iINwAOeZm36xdyJ4qsJuRqp55CncmVLsVHYJh8W8nYA=; h=To:From:Subject:Date; b=YKDWJ90B+1A1jAKYPZ7kGjoMx4medN6iTxuosx+sdLWnT7VGKM2jWiKbBsTSFN8fM Sj5LEwTFx6Odmu1wE/Zju04t5aj8G43Xn+U6y0sUPza3Hi9vACWmmcR+3F/etaC6iA fHf4Yo1G6a4llxmrcLtJkV9hy4x4/+4B5B7+6bf39jmXsdAiL02UmL99W8vOHfC3p9 /IzcnQhrRcKRGwF8AlgRnppNhRH8bIJPV8ZZjs+zdiYF7HJfKgYC8Mq/y2ZDQ+ZPNz +mzlrOrpVWVL4Ks2G7iyKiIl4Kh7QkE7s1YkV+APsLJHnospEiCyPt3EAu6DL5k0MW 21J290HUKXtww== From: geoffroy desvernay Subject: mpr(4) bug ? Openpgp: id=E15095B3F06A1012EE2921023FCFF4094587A0F0 Message-ID: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> Date: Mon, 12 Dec 2016 15:30:24 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="s8UhIMLDtkRFbXCvK8eIKJpp82atFihbq" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 14:37:09 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --s8UhIMLDtkRFbXCvK8eIKJpp82atFihbq Content-Type: multipart/mixed; boundary="0urjAb0MDHMBCqoq3LjLq58VcFsBOmuhk"; protected-headers="v1" From: geoffroy desvernay To: FreeBSD-scsi Message-ID: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> Subject: mpr(4) bug ? --0urjAb0MDHMBCqoq3LjLq58VcFsBOmuhk Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi all, First, I'm not fluently speaking SCSI nor kernel-c, so please don't byte too hard if I'm missing something obvious :) I tried some thing before posting here, from testing the hardware under linux (it work flawlessly there) and vendor's tests software, changing the adapter (for a different one, but with same chipset, that's all I have), upgrading firmwares where available (card and dell enclosure), trying to read mpr(4)'s code=E2=80=A6 well this is beyond my knowledge. Hardware: dell PowerEgde R430 with an LSI SAS3008 card and an MD1420 enclosure with 24 2T Seagate sas drives. This machine also have an embedded SAS3008 (dell perc H330) in non-raid mode (mrsas driver) with 4 SSD drives to be used as ZFS cache/log=E2=80=A6= System: FreeBSD 11.0-RELEASE-p3 Please tell me if there are tests I could do, patches to try, or ? Currently compiling 11-STABLE kernel with sys/dev/mpr from CURRENT, but with no clues=E2=80=A6 #pciconf -lv: mpr0@pci0:4:0:0: class=3D0x010700 card=3D0x1f461028 chip=3D0x00971= 000 rev=3D0x02 hdr=3D0x00 vendor =3D 'LSI Logic / Symbios Logic' device =3D 'SAS3008 PCI-Express Fusion-MPT SAS-3' class =3D mass storage subclass =3D SAS Symptoms: any zpool create fails with # zpool create ztest raidz da20 da21 da22 cannot create 'ztest': invalid argument for this pool operation # dmesg show a buncf of messages like this one: (da22:mpr0:0:26:0): READ(10). CDB: 28 00 e8 e0 88 af 00 00 01 00 (da22:mpr0:0:26:0): CAM status: SCSI Status Error (da22:mpr0:0:26:0): SCSI status: Check Condition (da22:mpr0:0:26:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) (da22:mpr0:0:26:0): Error 22, Unretryable error (see http://dgeo.perso.ec-m.fr/mpr_fail.txt for full related dmesg) # camcontrol devlist 2 seems normal to me: at scbus1 target 0 lun 0 (pass0,da0) at scbus1 target 1 lun 0 (pass1,da1) at scbus1 target 2 lun 0 (pass2,da2) at scbus1 target 3 lun 0 (pass3,da3) at scbus1 target 32 lun 0 (pass4,ses0)= at scbus2 target 8 lun 0 (pass5,da4) at scbus2 target 9 lun 0 (pass6,da5) at scbus2 target 10 lun 0 (pass7,da6) at scbus2 target 11 lun 0 (pass8,da7) at scbus2 target 12 lun 0 (pass9,da8) at scbus2 target 13 lun 0 (pass10,da9)= at scbus2 target 14 lun 0 (pass11,da10= ) at scbus2 target 15 lun 0 (pass12,da11= ) at scbus2 target 16 lun 0 (pass13,da12= ) at scbus2 target 17 lun 0 (pass14,da13= ) at scbus2 target 18 lun 0 (pass15,da14= ) at scbus2 target 19 lun 0 (pass16,da15= ) at scbus2 target 20 lun 0 (pass17,da16= ) at scbus2 target 21 lun 0 (pass18,da17= ) at scbus2 target 22 lun 0 (pass19,da18= ) at scbus2 target 23 lun 0 (pass20,da19= ) at scbus2 target 24 lun 0 (pass21,da20= ) at scbus2 target 25 lun 0 (pass22,da21= ) at scbus2 target 26 lun 0 (pass23,da22= ) at scbus2 target 27 lun 0 (pass24,da23= ) at scbus2 target 28 lun 0 (pass25,da24= ) at scbus2 target 29 lun 0 (pass26,da25= ) at scbus2 target 30 lun 0 (pass27,da26= ) at scbus2 target 31 lun 0 (pass28,da27= ) at scbus2 target 32 lun 0 (pass29,ses1= ) at scbus7 target 0 lun 0 (pass30,ses2)= With dev.mpr.0.debug_level: 1023, I tried a simple dd test: dd reports success if bs < 127k; fails if >=3D 128k (in both tests there are ILLEGAL= REQUEST in logs): dd if=3D/tmp/rnd of=3D/dev/da20 bs=3D127k: http://dgeo.perso.ec-m.fr/dd_bs_127k.debug.log bs=3D128k: http://dgeo.perso.ec-m.fr/dd_bs_128k.debug.log --=20 *geoffroy desvernay* C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de Marseille Tel: (+33|0)4 91 05 45 24 Fax: (+33|0)4 91 05 44 26 --0urjAb0MDHMBCqoq3LjLq58VcFsBOmuhk-- --s8UhIMLDtkRFbXCvK8eIKJpp82atFihbq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE4VCVs/BqEBLuKSECP8/0CUWHoPAFAlhOtIAACgkQP8/0CUWH oPDXTQ/+MLSK9T8f1yWAYUfViXERMMt3Hp5wL0MLgqD5iDxsRWKNbV14NHma/z++ qCsCK1sGtYhMJy4wao/MWK6e8aqh55Qfgcmh55yCPBPbMIWvMmxWZZUFh0G0QWw7 EXWHT/ZCnfAEEJDeSEiZ6zkENLxWcr6bEP9UbsNMKbOZhaXzaHO96oxQrMEQsbHI t5u5a7b0262GbnMm5CN3cFyOr1qo75QN5nuJ+UVGe7nhUY2RSVe+XvdNUJB/9lzM D8vvDPykBys0EjYXbETanzQCuTrvNUcAqN2WctQZnh6qOJDA3ZdieYtvgQlLblFO 5b3nACK638VqUQiMhXlhSlK0Sns3vbPR6fJ3B+xULbgF8oKtqEV+vsL0owJ5nTvQ Fz9AjhlYZEr9y75QGuhR26smBX5I9MM1KlD9xvCxY1Olsag6tAjBbsBPGhbTHqcu RAzeNWqt6YLtLvMDmbna6kKyrPpmL0eIgocZTFHR0TXU/CDfu74+uwRYZfAmZAbN vjLH7wiS7b0ZlTA9fHp9WZXagz5u/sxDZUfpJbSnuF3GLM0eVoMVXh+uYAaRx5wG kamopfNwqQCoN52QoG7t0eqv0AHad1PVnY5YqrWMnAYlO7z5ArKZMiksu+PMrVfO yI77hllWtE7OuiiZRIXBaLCWaRsk6IDt6dscWX5NP3LsVsYdnXg= =DlcC -----END PGP SIGNATURE----- --s8UhIMLDtkRFbXCvK8eIKJpp82atFihbq-- From owner-freebsd-scsi@freebsd.org Wed Dec 14 17:49:07 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43222C80727 for ; Wed, 14 Dec 2016 17:49:07 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Received: from mel1.ec-m.fr (melout.ec-m.fr [147.94.19.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F379178D for ; Wed, 14 Dec 2016 17:49:06 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Subject: Re: mpr(4) bug ? DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=centrale-marseille.fr; s=smtp0; t=1481737737; bh=FTtoMcmEYf+grGiyqchH9w5OkaXQgeHLOCHaX9j4Xns=; h=Subject:To:References:From:Date:In-Reply-To; b=sHJy4blIZGTD2hVhBM+bBDHbymYThKa2CI9iIoRuSEyP4kWZgHd9rq2nTjUO2RzxO mhTWvA20jP9HjWgGWsQkOEhWznQrT9Jub/UIyDOn48f4lZE8dvXVKqg0PJle3SwKF+ 33/zU4/mACX+zJHM+5NbD9RMvH4cKemw30cpm/oteQGRthXjlmU+ObdJS7cX4jvt3H vN4Z74eLfWZpGM8KDgzHVHWzUso6BH/hukAujs5qzpNFHvBLmSpL9e3YidJxlFDGbx Q8Ze0Vm95Ui5jOxdI9VHhjPzEzQvCGtYFEFNO73i2cTkw7QuMb7zgLTmfJkoKPiB5Z PyQgvX+H1I6cA== To: freebsd-scsi@freebsd.org References: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> From: geoffroy desvernay Message-ID: <95d92193-ba66-ba40-e417-01f29510e73c@centrale-marseille.fr> Date: Wed, 14 Dec 2016 18:48:55 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="bcP0XOxgjgj0V8cS1VQfvH6RnoFITDqge" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 17:49:07 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --bcP0XOxgjgj0V8cS1VQfvH6RnoFITDqge Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Did I say something silly, is there an obvious thing I'm missing ? Isn't it the right mailing list to post this ? The very same setup works flawlessly under debian or centos=E2=80=A6 Isn'= t there something to investigate ? Is there someone from Avago around ? Thank you for reading=E2=80=A6 On 12/12/2016 03:30 PM, geoffroy desvernay wrote: > Hi all, >=20 > First, I'm not fluently speaking SCSI nor kernel-c, so please don't byt= e > too hard if I'm missing something obvious :) >=20 > I tried some thing before posting here, from testing the hardware under= > linux (it work flawlessly there) and vendor's tests software, changing > the adapter (for a different one, but with same chipset, that's all I > have), upgrading firmwares where available (card and dell enclosure), > trying to read mpr(4)'s code=E2=80=A6 well this is beyond my knowledge.= >=20 > Hardware: dell PowerEgde R430 with an LSI SAS3008 card and an MD1420 > enclosure with 24 2T Seagate sas drives. > This machine also have an embedded SAS3008 (dell perc H330) in non-rai= d > mode (mrsas driver) with 4 SSD drives to be used as ZFS cache/log=E2=80= =A6 >=20 > System: FreeBSD 11.0-RELEASE-p3 >=20 > Please tell me if there are tests I could do, patches to try, or ? > Currently compiling 11-STABLE kernel with sys/dev/mpr from CURRENT, but= > with no clues=E2=80=A6 >=20 > #pciconf -lv: > mpr0@pci0:4:0:0: class=3D0x010700 card=3D0x1f461028 chip=3D0x009= 71000 > rev=3D0x02 hdr=3D0x00 > vendor =3D 'LSI Logic / Symbios Logic' > device =3D 'SAS3008 PCI-Express Fusion-MPT SAS-3' > class =3D mass storage > subclass =3D SAS >=20 > Symptoms: any zpool create fails with > # zpool create ztest raidz da20 da21 da22 > cannot create 'ztest': invalid argument for this pool operation >=20 > # dmesg show a buncf of messages like this one: > (da22:mpr0:0:26:0): READ(10). CDB: 28 00 e8 e0 88 af 00 00 01 00 > (da22:mpr0:0:26:0): CAM status: SCSI Status Error > (da22:mpr0:0:26:0): SCSI status: Check Condition > (da22:mpr0:0:26:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid > command operation code) > (da22:mpr0:0:26:0): Error 22, Unretryable error >=20 > (see http://dgeo.perso.ec-m.fr/mpr_fail.txt for full related dmesg) >=20 > # camcontrol devlist 2 seems normal to me: > at scbus1 target 0 lun 0 (pass0,da0)= > at scbus1 target 1 lun 0 (pass1,da1)= > at scbus1 target 2 lun 0 (pass2,da2)= > at scbus1 target 3 lun 0 (pass3,da3)= > at scbus1 target 32 lun 0 (pass4,ses= 0) > at scbus2 target 8 lun 0 (pass5,da4)= > at scbus2 target 9 lun 0 (pass6,da5)= > at scbus2 target 10 lun 0 (pass7,da6= ) > at scbus2 target 11 lun 0 (pass8,da7= ) > at scbus2 target 12 lun 0 (pass9,da8= ) > at scbus2 target 13 lun 0 (pass10,da= 9) > at scbus2 target 14 lun 0 (pass11,da= 10) > at scbus2 target 15 lun 0 (pass12,da= 11) > at scbus2 target 16 lun 0 (pass13,da= 12) > at scbus2 target 17 lun 0 (pass14,da= 13) > at scbus2 target 18 lun 0 (pass15,da= 14) > at scbus2 target 19 lun 0 (pass16,da= 15) > at scbus2 target 20 lun 0 (pass17,da= 16) > at scbus2 target 21 lun 0 (pass18,da= 17) > at scbus2 target 22 lun 0 (pass19,da= 18) > at scbus2 target 23 lun 0 (pass20,da= 19) > at scbus2 target 24 lun 0 (pass21,da= 20) > at scbus2 target 25 lun 0 (pass22,da= 21) > at scbus2 target 26 lun 0 (pass23,da= 22) > at scbus2 target 27 lun 0 (pass24,da= 23) > at scbus2 target 28 lun 0 (pass25,da= 24) > at scbus2 target 29 lun 0 (pass26,da= 25) > at scbus2 target 30 lun 0 (pass27,da= 26) > at scbus2 target 31 lun 0 (pass28,da= 27) > at scbus2 target 32 lun 0 (pass29,se= s1) > at scbus7 target 0 lun 0 (pass30,ses= 2) >=20 > With dev.mpr.0.debug_level: 1023, I tried a simple dd test: dd reports > success if bs < 127k; fails if >=3D 128k (in both tests there are ILLEG= AL > REQUEST in logs): > dd if=3D/tmp/rnd of=3D/dev/da20 bs=3D127k: > http://dgeo.perso.ec-m.fr/dd_bs_127k.debug.log >=20 > bs=3D128k: http://dgeo.perso.ec-m.fr/dd_bs_128k.debug.log >=20 >=20 --=20 geoffroy desvernay C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de Marseille Tel: (+33|0)4 91 05 45 24 Fax: (+33|0)4 91 05 45 98 dgeo@centrale-marseille.fr --bcP0XOxgjgj0V8cS1VQfvH6RnoFITDqge Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYUYYIAAoJED/P9AlFh6DwJv4QALXgoiSVhMWpDXTxMMBJwGl6 OsZ1Yo92Luv9dnrjROB/dgotoYBaxI5wF/UptuBg4j3tphbLlhSjuuCmfhMs1mFT +0+c9rAuZOxOuPQ6E66PUEVhdig/1pycgVUO4exY1O/F0BKiRl0FqKN5tKrDpZpC 5x8VBa+8G1tsvdj94dErE3wI5wBGxBm7CnStWlh7c/p0Cusfrqzff87eSRTMZUfq GCpF3JGophpC1oe0YsCXr4cfSLJdjyI60etFKj7OO8AQKVoTORH4/QYUeM/Pg5ot 6O4BGYyLhAWysfZb3KsffFgBmpGketMjx9o2A8GY2oCjgKLARUkUBARhEy7zXgRb KbqkHpAr6oYHiOvJQKTulcckOEN6ioi269eMAHs2Rwsrtc1498iZ8dl6bPfdDRKk xAheNVpfQk57Y6Ayd0fZWo3GRUlrgmoh0qDL04tYT9c6MrteAvwBJpOBKNbRN5ia ewUa0lDdwdwutViJYdVonR0OcxGmJpTHFs6oIyaeXByAWehOD4wHwXjVZlaXSrlg 5w4SevPvl/g6PDJLF/+q4QiBGNy3aGvatCvOeLNG5wddTfFrXWo7irblUAGeOyCc 8Ri8w/myd6U8TMWY8uzmBSrUvPpnvsCnT3Th546ZyKAGAt6Vho/a4qPRP57SHfmS ZDmJK//fXTsLNWP2Yjli =xkIO -----END PGP SIGNATURE----- --bcP0XOxgjgj0V8cS1VQfvH6RnoFITDqge-- From owner-freebsd-scsi@freebsd.org Wed Dec 14 18:36:25 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20DFDC76A23 for ; Wed, 14 Dec 2016 18:36:25 +0000 (UTC) (envelope-from stephen.mcconnell@broadcom.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F08EEB83 for ; Wed, 14 Dec 2016 18:36:24 +0000 (UTC) (envelope-from stephen.mcconnell@broadcom.com) Received: by mail-it0-x22b.google.com with SMTP id c20so1368280itb.0 for ; Wed, 14 Dec 2016 10:36:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:content-transfer-encoding; bh=LvRp64nKpZnHkRxEbQgdwX3XZbqcFaC7DoCgG6Uf4pQ=; b=LUMQ+h/rIXkLt7n3fvWTNVf0x8dYGHIDuUB96Uq+Tavl3/ct7WBcbNiXRWcT2AMRJ4 oi5dOkp3aHGBNkCv6irn/JjV1qvZJhXxacxRlXYEgpPMpPoo7rn04gxB4Q5F25dbazwK H51OlGnmm3xr/xVELsevnCMtcpIg70At4ouMA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:content-transfer-encoding; bh=LvRp64nKpZnHkRxEbQgdwX3XZbqcFaC7DoCgG6Uf4pQ=; b=PVnzJnJBU+zzrWdWvQ0Itkat3UEMfwFUDM04xm/+GtmsmAGKqvzGD56kf/4hMvZ7sL ejitSc0EusS58Y6vxoB5QBcojlT6B80ovCU5tnhhviQ3cj8bB8y+vfc9EnNWvhsJJG8c IYFP9kXm+kopPsLvFY39kv2lsWXUyx+C8DSRPISZuePo+qCblyCduxms5acMJUtL147p s0x36jNxyXeUYTEeTGACaaumxH5ut/beSb/X3rrbyYIX3w74EjKzzPWBNbJRdMmRy7Ox ePKapI57FHptRdWwfHBm2HHUbdd3NN4R+c09lCOBhm/iG4qptGPCGY6beWybUPyEmKFV 76Mw== X-Gm-Message-State: AKaTC01D3TduuCTPMZ7kBY2MbZtRgUAbF7cQuTa9qIz90cTc2KCy5inAW//yrjK0vaXF7KSxXa1I04EUo1pzwYt1 X-Received: by 10.36.50.78 with SMTP id j75mr8589468ita.58.1481740584154; Wed, 14 Dec 2016 10:36:24 -0800 (PST) From: Stephen Mcconnell References: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> <95d92193-ba66-ba40-e417-01f29510e73c@centrale-marseille.fr> In-Reply-To: <95d92193-ba66-ba40-e417-01f29510e73c@centrale-marseille.fr> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQMXgPL58A0xrdK0wmpOU8KBSptG4wKiRkB+nmhbLOA= Date: Wed, 14 Dec 2016 11:36:23 -0700 Message-ID: <5fcf45f2ccbf3b4b195bfc16164bc843@mail.gmail.com> Subject: RE: mpr(4) bug ? To: geoffroy desvernay , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 18:36:25 -0000 Hi Geoffroy, I looked through the logs. It's strange. I don't know why there would be sense data for 'Invalid OP Code' for the read(10)/write(10) commands. The driver looks like it's doing everything correctly. It's just passing up the error and the command fails. Can you retry with debug_level set to 0xFFFF? It might not give more info, but we can see. Steve > -----Original Message----- > From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd- > scsi@freebsd.org] On Behalf Of geoffroy desvernay > Sent: Wednesday, December 14, 2016 10:49 AM > To: freebsd-scsi@freebsd.org > Subject: Re: mpr(4) bug ? > > Did I say something silly, is there an obvious thing I'm missing ? Isn't > it the right > mailing list to post this ? > > The very same setup works flawlessly under debian or centos=E2=80=A6 Isn'= t there > something to investigate ? > > Is there someone from Avago around ? > > Thank you for reading=E2=80=A6 > > On 12/12/2016 03:30 PM, geoffroy desvernay wrote: > > Hi all, > > > > First, I'm not fluently speaking SCSI nor kernel-c, so please don't > > byte too hard if I'm missing something obvious :) > > > > I tried some thing before posting here, from testing the hardware > > under linux (it work flawlessly there) and vendor's tests software, > > changing the adapter (for a different one, but with same chipset, > > that's all I have), upgrading firmwares where available (card and dell > > enclosure), trying to read mpr(4)'s code=E2=80=A6 well this is beyond m= y > > knowledge. > > > > Hardware: dell PowerEgde R430 with an LSI SAS3008 card and an MD1420 > > enclosure with 24 2T Seagate sas drives. > > This machine also have an embedded SAS3008 (dell perc H330) in > > non-raid mode (mrsas driver) with 4 SSD drives to be used as ZFS > > cache/log=E2=80=A6 > > > > System: FreeBSD 11.0-RELEASE-p3 > > > > Please tell me if there are tests I could do, patches to try, or ? > > Currently compiling 11-STABLE kernel with sys/dev/mpr from CURRENT, > > but with no clues=E2=80=A6 > > > > #pciconf -lv: > > mpr0@pci0:4:0:0: class=3D0x010700 card=3D0x1f461028 chip=3D0x009= 71000 > > rev=3D0x02 hdr=3D0x00 > > vendor =3D 'LSI Logic / Symbios Logic' > > device =3D 'SAS3008 PCI-Express Fusion-MPT SAS-3' > > class =3D mass storage > > subclass =3D SAS > > > > Symptoms: any zpool create fails with > > # zpool create ztest raidz da20 da21 da22 cannot create 'ztest': > > invalid argument for this pool operation > > > > # dmesg show a buncf of messages like this one: > > (da22:mpr0:0:26:0): READ(10). CDB: 28 00 e8 e0 88 af 00 00 01 00 > > (da22:mpr0:0:26:0): CAM status: SCSI Status Error > > (da22:mpr0:0:26:0): SCSI status: Check Condition > > (da22:mpr0:0:26:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid > > command operation code) > > (da22:mpr0:0:26:0): Error 22, Unretryable error > > > > (see http://dgeo.perso.ec-m.fr/mpr_fail.txt for full related dmesg) > > > > # camcontrol devlist 2 seems normal to me: > > at scbus1 target 0 lun 0 (pass0,da0) > > at scbus1 target 1 lun 0 (pass1,da1) > > at scbus1 target 2 lun 0 (pass2,da2) > > at scbus1 target 3 lun 0 (pass3,da3) > > at scbus1 target 32 lun 0 > > (pass4,ses0) > > at scbus2 target 8 lun 0 (pass5,da4) > > at scbus2 target 9 lun 0 (pass6,da5) > > at scbus2 target 10 lun 0 (pass7,da6= ) > > at scbus2 target 11 lun 0 (pass8,da7= ) > > at scbus2 target 12 lun 0 (pass9,da8= ) > > at scbus2 target 13 lun 0 > > (pass10,da9) > > at scbus2 target 14 lun 0 > > (pass11,da10) > > at scbus2 target 15 lun 0 > > (pass12,da11) > > at scbus2 target 16 lun 0 > > (pass13,da12) > > at scbus2 target 17 lun 0 > > (pass14,da13) > > at scbus2 target 18 lun 0 > > (pass15,da14) > > at scbus2 target 19 lun 0 > > (pass16,da15) > > at scbus2 target 20 lun 0 > > (pass17,da16) > > at scbus2 target 21 lun 0 > > (pass18,da17) > > at scbus2 target 22 lun 0 > > (pass19,da18) > > at scbus2 target 23 lun 0 > > (pass20,da19) > > at scbus2 target 24 lun 0 > > (pass21,da20) > > at scbus2 target 25 lun 0 > > (pass22,da21) > > at scbus2 target 26 lun 0 > > (pass23,da22) > > at scbus2 target 27 lun 0 > > (pass24,da23) > > at scbus2 target 28 lun 0 > > (pass25,da24) > > at scbus2 target 29 lun 0 > > (pass26,da25) > > at scbus2 target 30 lun 0 > > (pass27,da26) > > at scbus2 target 31 lun 0 > > (pass28,da27) > > at scbus2 target 32 lun 0 > > (pass29,ses1) > > at scbus7 target 0 lun 0 > > (pass30,ses2) > > > > With dev.mpr.0.debug_level: 1023, I tried a simple dd test: dd reports > > success if bs < 127k; fails if >=3D 128k (in both tests there are > > ILLEGAL REQUEST in logs): > > dd if=3D/tmp/rnd of=3D/dev/da20 bs=3D127k: > > http://dgeo.perso.ec-m.fr/dd_bs_127k.debug.log > > > > bs=3D128k: http://dgeo.perso.ec-m.fr/dd_bs_128k.debug.log > > > > > > > -- > geoffroy desvernay > C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de Ma= rseille > Tel: (+33|0)4 91 05 45 24 > Fax: (+33|0)4 91 05 45 98 > dgeo@centrale-marseille.fr > From owner-freebsd-scsi@freebsd.org Wed Dec 14 18:57:31 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A9B1C8026D for ; Wed, 14 Dec 2016 18:57:31 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Received: from mel1.ec-m.fr (melout.ec-m.fr [147.94.19.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 60DD1187B for ; Wed, 14 Dec 2016 18:57:29 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Subject: Re: mpr(4) bug ? DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=centrale-marseille.fr; s=smtp0; t=1481741847; bh=YPehxk9vVBFMs0T2YN2pL+o6kNbC7APp9nUxvJ17ZLg=; h=Subject:To:References:From:Date:In-Reply-To; b=BdOuNm/8rfRGgxXNy0Sc1WLjgeykyAS5JS3LV3WUCO3NnSWWz9bBAw0hFFnsYQC5Z MgkWWsxSn5brA/zoXTuDnI4dNsNU+eISFHvECtjiAgZX0cnvwrVLsDvWBgiDCJP8XZ caEVW1oa5XODoIMe1FLWp83Bbrb/rpYikHvGrgnC1TAlGMp1uTao3OTPvL+NYYM0gj Z9c0LpIwM815mdkLSCCY2fK4puZbhovTah8Bn2Xsh8OrWaJLF/5xFuGizws0+H6ORN SdtEmNsxsLA5ZM+8I/+O71tgz00/Cygjd6/cOeLrbHiXBtgB8RHElivkKsS8zYyMab iutOidPwnTOuA== To: freebsd-scsi@freebsd.org References: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> <95d92193-ba66-ba40-e417-01f29510e73c@centrale-marseille.fr> <5fcf45f2ccbf3b4b195bfc16164bc843@mail.gmail.com> From: geoffroy desvernay Message-ID: <489d892d-7734-e148-98d3-77bc6e62bac1@centrale-marseille.fr> Date: Wed, 14 Dec 2016 19:57:25 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: <5fcf45f2ccbf3b4b195bfc16164bc843@mail.gmail.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QVQ1peRRwoqsogWWcEokJIip6TJLXM4tD" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 18:57:31 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --QVQ1peRRwoqsogWWcEokJIip6TJLXM4tD Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 12/14/2016 07:36 PM, Stephen Mcconnell via freebsd-scsi wrote: > Hi Geoffroy, >=20 > I looked through the logs. It's strange. I don't know why there would b= e > sense data for 'Invalid OP Code' for the read(10)/write(10) commands. T= he > driver looks like it's doing everything correctly. It's just passing up= the > error and the command fails. Can you retry with debug_level set to 0xFF= FF? > It might not give more info, but we can see. >=20 > Steve >=20 Thank you :) here is a dmesg exerpt, after sysctl dev.mpr.0.debug_level=3D0xFFFF dd if=3D/dev/zero of=3D/dev/da20 bs=3D1M and http://dgeo.perso.ec-m.fr/mfi0_dd_dmesg.log Do you prefer bs=3D127k or bs=3D128k ? --=20 geoffroy desvernay C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de Marseille Tel: (+33|0)4 91 05 45 24 Fax: (+33|0)4 91 05 45 98 dgeo@centrale-marseille.fr --QVQ1peRRwoqsogWWcEokJIip6TJLXM4tD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYUZYWAAoJED/P9AlFh6DwW7UQAItnHyZfYIVnDpnzN7jZDYjp 4Fy+Uaf2SgjE0o2qCA75YCt3DZYr+GItO5Qy2o2vbmgQYhNDTnLSaxS/x5YMy2wV mFlBW4XxR3wRERJpgu8ILcZFyONHXEKmAUGDx2c5AfvBTguCEUq5GvfC7WF9X7Y8 +IVLbQyjLPJvvNmQfRc+9UqYc2OcRBrpxhj4LJ5sZWeQUztiBDnAZFV2AAkUtzda v9Ac5BQsN75B1+5WqmCU7t27WoRGUCOEFqeBJLkBVd4R6rh1cvMnEA13He1o04d+ G4faRCAT30gO73DAG+IYiLy03GukXAfOa/Iw0thi6d2FW4IGNvqXzMOBpGnM20WY E2w2xfHgbBnaih9fKk79WyGwwPcL0AxfthPrhyZ7Xni0qDMICXLuMFKJVcOcbxeo JN/49kV5FTuCpsOGHrUkkAkIy8DfpdUgdbP3ovZTOJa2Oz3Xw3DImiS3eY7A5bkt UN37z+dRj2CfAeuQN4XG6hdZDI2I77lO5+8d/lJmuHOWITlszagdxdQ48AIC5Vvi CruSCKokeiG61PGDJYEutt36nuRiyFibPKRAS0B+Y6xeeTw72AqvHnaPD9fJdTTU psZ1jf2rIOutWV1CpcSKb7Gd1aAN3GTZWlz9a8LtxVs2RHRsZkPVOf1ZPjZWqiIr MkAUpWgUTGpcVcXP2za5 =mCdQ -----END PGP SIGNATURE----- --QVQ1peRRwoqsogWWcEokJIip6TJLXM4tD-- From owner-freebsd-scsi@freebsd.org Wed Dec 14 19:06:49 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F07FC807C5 for ; Wed, 14 Dec 2016 19:06:49 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Received: from mel0.ec-m.fr (melout.ec-m.fr [147.94.19.37]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E721C66D for ; Wed, 14 Dec 2016 19:06:48 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Subject: Re: mpr(4) bug ? DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=centrale-marseille.fr; s=smtp0; t=1481741988; bh=jI3lE2vcEqlncngrff6B/Xx50h22W0mEh5CJ5vlQjro=; h=Subject:To:References:From:Date:In-Reply-To; b=jmummLu/dp//8py72UEfxbZA1nCdKwLna6CRAed1PJgvs5+Xvtb5Kwqs8TDC6e5ot jKwkPR6qIeatVMADUnPhZqDwdvxH0LVF62dEHxxWFKJW+yHlGV0cMsXAjIrKuH2egU Xb2vcKlNllalfs8KyNb3r3eZ5Ev0F6ZG9KFLRfoJhsV7rwqv44gkfzIeFYfl88Ivo1 u4vMO3GzcXEJPs8ba0RH04seBdQRGNS6gsGUeKEQRXgHPDGJOA0Txb/Fyas769+9Ue yp3ngcuWyUQwQnVEsl6mxzzZUm1En4Dtvt8Azkg0nUW+XZbthKi6n59sMXVkrsARyC 3B1Z7jVHl/Nhg== To: freebsd-scsi@freebsd.org References: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> <95d92193-ba66-ba40-e417-01f29510e73c@centrale-marseille.fr> <5fcf45f2ccbf3b4b195bfc16164bc843@mail.gmail.com> <489d892d-7734-e148-98d3-77bc6e62bac1@centrale-marseille.fr> From: geoffroy desvernay Message-ID: <8d5ee1d2-e7c4-8a3f-5f8d-b1ef29995e9b@centrale-marseille.fr> Date: Wed, 14 Dec 2016 19:59:47 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: <489d892d-7734-e148-98d3-77bc6e62bac1@centrale-marseille.fr> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="xRsg6P7gXQXLIfPnu2sQFILHpkiLk5CKJ" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 19:06:49 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --xRsg6P7gXQXLIfPnu2sQFILHpkiLk5CKJ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Oups, just overwritten it: it was truncated=E2=80=A6 the new may be more complete=E2=80=A6 On 12/14/2016 07:57 PM, geoffroy desvernay wrote: > On 12/14/2016 07:36 PM, Stephen Mcconnell via freebsd-scsi wrote: >> Hi Geoffroy, >> >> I looked through the logs. It's strange. I don't know why there would = be >> sense data for 'Invalid OP Code' for the read(10)/write(10) commands. = The >> driver looks like it's doing everything correctly. It's just passing u= p the >> error and the command fails. Can you retry with debug_level set to 0xF= FFF? >> It might not give more info, but we can see. >> >> Steve >> >=20 > Thank you :) >=20 > here is a dmesg exerpt, after >=20 > sysctl dev.mpr.0.debug_level=3D0xFFFF > dd if=3D/dev/zero of=3D/dev/da20 bs=3D1M and >=20 > http://dgeo.perso.ec-m.fr/mfi0_dd_dmesg.log >=20 > Do you prefer bs=3D127k or bs=3D128k ? >=20 --=20 geoffroy desvernay C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de Marseille Tel: (+33|0)4 91 05 45 24 Fax: (+33|0)4 91 05 45 98 dgeo@centrale-marseille.fr --xRsg6P7gXQXLIfPnu2sQFILHpkiLk5CKJ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYUZajAAoJED/P9AlFh6Dwt3AP/j7RxHlpa2QkwYhvXLS75sKL wJH1BVv+vdnDey/HCdmkFYXPXqF95ClUG630v2qloyQ8kzqFuV7WD+TVlmVYEKcG snLgED9HLtEePhyzO8PJtvBPb+eUZWBIbZ3Uo8LRTRWEOHKCGa7fMAB5ake/xR+q o9lP5BCTeO2f4SSRAXgsDVPYfBFyLwZt4U9X4cgSpvraMM+bo4M21qbt6tgYeABY kIqM5TyhH+J/22G0xMLSvdZafku3YnIAJEDK9ml1kH/CfxxDO2r/O4VNe3eG3ZQU 1iyPfx2kZMCEqQGpsai9UZhWenCby6/x6ixqdEweDDpFLVGfuf/wWS0qzVYiXLII CgoogjGS7olomSbPGQlfVcUgx0pSSuI8h1dJRMxNTKOSxJfUwARuGLIgTnQCSEz4 f9q+59eEPdISYWWhzqB9CNvGxvadeVs244d5fuwQYZJQIy5xsY083TVrT6eFaiKk trA4WgwvfglRZjqnAehNzjuZAamW3y/0Z2ZGAqMyNeSuRKWWeSAxqyZJnu31vgjE Wu/PCIWutJH4nQ34RMMQT2hQq0JXyAAk1NNwavtCbDIKvSegOdnkGktM8G2aIM9i T0z/nSv1Shk2oEqarOORj03o6wd/P1D3wqokH74/S+pYW3ws55EUkqSstQ8D5VCt CdbfqaP5ZU+J0P9DjsjN =b8RA -----END PGP SIGNATURE----- --xRsg6P7gXQXLIfPnu2sQFILHpkiLk5CKJ-- From owner-freebsd-scsi@freebsd.org Wed Dec 14 19:44:47 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D68C9C806DE for ; Wed, 14 Dec 2016 19:44:47 +0000 (UTC) (envelope-from stephen.mcconnell@broadcom.com) Received: from mail-io0-x231.google.com (mail-io0-x231.google.com [IPv6:2607:f8b0:4001:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8292121F for ; Wed, 14 Dec 2016 19:44:47 +0000 (UTC) (envelope-from stephen.mcconnell@broadcom.com) Received: by mail-io0-x231.google.com with SMTP id d9so49764604ioe.0 for ; Wed, 14 Dec 2016 11:44:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to:content-transfer-encoding; bh=2myU5m61HDgl3BR4TYXEPXayv+DClXw3gORzFeC+EI4=; b=T6feymPTjWOxqJ0WTrI8p/ebkEfUBcU9du2NFY4El1ljvHrHfnSgPQvuo3PGhJ4Umh jkxmkc9hTFylDoljhFWSvVp1apjOaAvYQCWf93O9eaEjrfrf4cFhvOYKAN8SJ06bslF7 2ENgtnOQUuRQgPnJ+0pjIAuDhF8hvycnfihHM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to:content-transfer-encoding; bh=2myU5m61HDgl3BR4TYXEPXayv+DClXw3gORzFeC+EI4=; b=B7/KztheYi1zuJHjirAyYg2TAsQa3xVXyVAYAwb5FDMJltL3oHgg7FiI+cyKYUeaMJ U5RO5or45JODvPAkmysTz1eXToa6NPlnbPyiQd3Q1/YCSf0WAyG9zZI5y6Frheaivw2t BRAHbLJM4EA27/VykGM3WFZJ6oigThIvCH0Wa/9UbwoTISdt5lMEbE5V+3n5lwmuW2BD KcFBCYHU/rKVjMICKT91OAY2jp6ykn28VeukrBbW/VTYpyNMY5KrQmiGcl66v8S4yU0Q So1+EFxRn+Rj0KgMijE8Zu2suN/nDOaCcyNbqBAdskTloXKTQcggfrDoWN2KB8FFHD7B EGUw== X-Gm-Message-State: AKaTC019H6t/OpKqHHbh0tnQo/0fqf/jUTBx5DEjbuetL8WcK8R1ocrkaYLf1BbOyFjzzpEMHvZaNxyHq8/POPQ5 X-Received: by 10.107.162.132 with SMTP id l126mr94594289ioe.37.1481744686828; Wed, 14 Dec 2016 11:44:46 -0800 (PST) From: Stephen Mcconnell References: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> <95d92193-ba66-ba40-e417-01f29510e73c@centrale-marseille.fr> <5fcf45f2ccbf3b4b195bfc16164bc843@mail.gmail.com> <489d892d-7734-e148-98d3-77bc6e62bac1@centrale-marseille.fr> <8d5ee1d2-e7c4-8a3f-5f8d-b1ef29995e9b@centrale-marseille.fr> In-Reply-To: <8d5ee1d2-e7c4-8a3f-5f8d-b1ef29995e9b@centrale-marseille.fr> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQMXgPL58A0xrdK0wmpOU8KBSptG4wKiRkB+AhxmKYIChOrSJgIYj7cQnjKd3lA= Date: Wed, 14 Dec 2016 12:44:45 -0700 Message-ID: <853ce95f576625f247b625268a8a8b98@mail.gmail.com> Subject: RE: mpr(4) bug ? To: geoffroy desvernay , freebsd-scsi@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 19:44:47 -0000 I don't see anything new that helps. Again, the driver looks like it's operating properly - just reporting the error that it gets. All I can think of is that the requested LBA is out of range, but if that were that case yo= u should be getting a different error (ASC of 21 instead of 20). And one othe= r thing I see is that the debug output from MPR only shows errors for da20:mpr0:0:24:0. The errors at the beginning of the log show several drives, so I'm not sure what to make of that. Are you using the same disks for FreeBSD as you are for Linux? If not, mayb= e you can do that and see what happens. Steve > -----Original Message----- > From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd- > scsi@freebsd.org] On Behalf Of geoffroy desvernay > Sent: Wednesday, December 14, 2016 12:00 PM > To: freebsd-scsi@freebsd.org > Subject: Re: mpr(4) bug ? > > Oups, just overwritten it: it was truncated=E2=80=A6 the new may be more = complete=E2=80=A6 > > On 12/14/2016 07:57 PM, geoffroy desvernay wrote: > > On 12/14/2016 07:36 PM, Stephen Mcconnell via freebsd-scsi wrote: > >> Hi Geoffroy, > >> > >> I looked through the logs. It's strange. I don't know why there would > >> be sense data for 'Invalid OP Code' for the read(10)/write(10) > >> commands. The driver looks like it's doing everything correctly. It's > >> just passing up the error and the command fails. Can you retry with > debug_level set to 0xFFFF? > >> It might not give more info, but we can see. > >> > >> Steve > >> > > > > Thank you :) > > > > here is a dmesg exerpt, after > > > > sysctl dev.mpr.0.debug_level=3D0xFFFF > > dd if=3D/dev/zero of=3D/dev/da20 bs=3D1M and > > > > http://dgeo.perso.ec-m.fr/mfi0_dd_dmesg.log > > > > Do you prefer bs=3D127k or bs=3D128k ? > > > > > -- > geoffroy desvernay > C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de Ma= rseille > Tel: (+33|0)4 91 05 45 24 > Fax: (+33|0)4 91 05 45 98 > dgeo@centrale-marseille.fr > From owner-freebsd-scsi@freebsd.org Wed Dec 14 20:45:28 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89AB1C775A5 for ; Wed, 14 Dec 2016 20:45:28 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Received: from mel1.ec-m.fr (melout.ec-m.fr [147.94.19.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3DD3816A4 for ; Wed, 14 Dec 2016 20:45:27 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Subject: Re: mpr(4) bug ? DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=centrale-marseille.fr; s=smtp0; t=1481748325; bh=2Y+HIKlXaJJ2dObVEZQ9R+iEChPqYESnMjbkPViMSC0=; h=Subject:To:References:From:Date:In-Reply-To; b=k/f2p8rF/ztubhHvTtHD2GkceAQKuijixf5rzyp5EMS/mDA18B1s8euC2RwqWS5C2 NqwdOcmIuivh577MHaliWnAIlLVJA1mhvjejTeSKgqLVRaMhtFHAfrNi1mnemt/ul4 W1ZOKiRBMQ9QUFzpO5Jtq5DLxDkgJST+CPdsRKqibWsD2ZigdNztIf2vimYj6QDjXm iYrG9FJxR4GH1pE2v+HfEJ8CrKQM9Paut5iLPqv64co0UyM+86+wY8ThTaaRveZzlB 71zyz/42c30qC+NZ16O7vUyS6FfLlp/JW5jSYt5TYs4G4HHdPQa6Y7WvnizjRVCwmt EWE8gkwXCrdIw== To: Stephen Mcconnell , freebsd-scsi@freebsd.org References: <2ae74eaa-80da-2b81-900b-9b9d21080e5c@centrale-marseille.fr> <95d92193-ba66-ba40-e417-01f29510e73c@centrale-marseille.fr> <5fcf45f2ccbf3b4b195bfc16164bc843@mail.gmail.com> <489d892d-7734-e148-98d3-77bc6e62bac1@centrale-marseille.fr> <8d5ee1d2-e7c4-8a3f-5f8d-b1ef29995e9b@centrale-marseille.fr> <853ce95f576625f247b625268a8a8b98@mail.gmail.com> From: geoffroy desvernay X-Enigmail-Draft-Status: N1110 Message-ID: Date: Wed, 14 Dec 2016 21:45:24 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Icedove/45.4.0 MIME-Version: 1.0 In-Reply-To: <853ce95f576625f247b625268a8a8b98@mail.gmail.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="WSsksJ2MmDkORlvCVTjLliT0BeonjHg4D" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2016 20:45:28 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WSsksJ2MmDkORlvCVTjLliT0BeonjHg4D Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Steve, First of all, thank you for taking time with this problem ! Yes, I used the very same machine/controller/cable/enclosure/disks with a live centos system and to check it, and I installed a working debian system on the bay without problem=E2=80=A6 That's why I'm thinking the ke= y of my problem lives somewhere on src/sys ( well, I don't understand kernel/scsi/sas stuff enought to be sure of this :) The last dmesg were with a CURRENT kernel, I also tested with 11-STABLE and sys/dev/mpr from CURRENT and 11-RELEASE, it looks the same (I saw current mpr(4) seems to have few debugging changes recently) I'll try tomorrow to change one disk with other 2.5' SAS drive to see if they could be the culprits=E2=80=A6 these ones are "SEAGATE ST2000NX0433 = NS02" as reported by "camcontrol devlist", sold (modified ?) by dell. they behave correctly with smartctl, and the error does not block dd command if bs < 128k=E2=80=A6 not sure what it could mean. Anyway, thank you for your time, and I'd be happy to follow any advice helping to sort out this =E2=80=A6 (patch, live system, serial debug, =E2= =80=A6) Yours, Geoffroy. On 12/14/2016 08:44 PM, Stephen Mcconnell via freebsd-scsi wrote: > I don't see anything new that helps. Again, the driver looks like it's > operating properly - just reporting the error that it gets. All I can t= hink > of is that the requested LBA is out of range, but if that were that cas= e you > should be getting a different error (ASC of 21 instead of 20). And one = other > thing I see is that the debug output from MPR only shows errors for > da20:mpr0:0:24:0. The errors at the beginning of the log show several > drives, so I'm not sure what to make of that. >=20 > Are you using the same disks for FreeBSD as you are for Linux? If not, = maybe > you can do that and see what happens. >=20 > Steve >=20 >> -----Original Message----- >> From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd- >> scsi@freebsd.org] On Behalf Of geoffroy desvernay >> Sent: Wednesday, December 14, 2016 12:00 PM >> To: freebsd-scsi@freebsd.org >> Subject: Re: mpr(4) bug ? >> >> Oups, just overwritten it: it was truncated=E2=80=A6 the new may be mo= re complete=E2=80=A6 >> >> On 12/14/2016 07:57 PM, geoffroy desvernay wrote: >>> On 12/14/2016 07:36 PM, Stephen Mcconnell via freebsd-scsi wrote: >>>> Hi Geoffroy, >>>> >>>> I looked through the logs. It's strange. I don't know why there woul= d >>>> be sense data for 'Invalid OP Code' for the read(10)/write(10) >>>> commands. The driver looks like it's doing everything correctly. It'= s >>>> just passing up the error and the command fails. Can you retry with >> debug_level set to 0xFFFF? >>>> It might not give more info, but we can see. >>>> >>>> Steve >>>> >>> >>> Thank you :) >>> >>> here is a dmesg exerpt, after >>> >>> sysctl dev.mpr.0.debug_level=3D0xFFFF >>> dd if=3D/dev/zero of=3D/dev/da20 bs=3D1M and >>> >>> http://dgeo.perso.ec-m.fr/mfi0_dd_dmesg.log >>> >>> Do you prefer bs=3D127k or bs=3D128k ? >>> >> >> >> -- >> geoffroy desvernay >> C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de= Marseille >> Tel: (+33|0)4 91 05 45 24 >> Fax: (+33|0)4 91 05 45 98 >> dgeo@centrale-marseille.fr >> > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org"= >=20 --=20 geoffroy desvernay C.R.I - Administration syst=C3=A8mes et r=C3=A9seaux Ecole Centrale de Marseille Tel: (+33|0)4 91 05 45 24 Fax: (+33|0)4 91 05 45 98 dgeo@centrale-marseille.fr --WSsksJ2MmDkORlvCVTjLliT0BeonjHg4D Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYUa9kAAoJED/P9AlFh6DwpVIP/1fEwMNc9GlcXa39PzQicLMc Wm0t/YZsscDw95KxPgGbP1LUWBJXy8XV0wjBoR8ahCQy/loFD8mTo/nFDMwKKv3b MeClubovjLY76Cmsx2P/echG+UpzrlGauN1wmlXr+ZgLCAOtO+vbWExIQ0FPs/Q/ OAbRS4R3NimXrcog1YdcMJjjniGph6UPSvJz11Kd26wpQj09tYOYzRgMlqBk49TV oICXgXCz+9l5NaMgIZAdz+yAxUz3WiFblscI3aizaL5Ic3vBQ5E20RhOcfT+k1yB wzNU7PfDFrOkMShBNU+poalt5MUdygl+2ldNJXG3R/i7kBv2Ag6tru71mqAAvBj6 k/JKtCFW3ZfIV/BfW+YH4oGT9Lwz2J+wkkqoUZCvj50Pxk6247dwkkvdrADB0ioS 9BVjpd6S+qgK/0eGEaQVFz3Th+h4xdu3gJi8AI0GlRySeh/dTyc9kzi/ce6y3wSd fOE8z+8lERbPoecZXx9+yu/ohvjvFpHZMsU5i9OfCDlX0pHUbTThMWXYGgAm9MhV YycC4E+f7qE/e6PdK3hWmZ0iTXqF8OlipcJHAT3P3MESIRVpt0LH0GC4Er/2sG5f cQtaTqlbEJ3rxD8dORff/JD0R3GUUk+/nwBFRXeKxWg0XTCp0T6dv+zTp7fhyNgk lteOUonfiFrMhUgbuoDa =uUs6 -----END PGP SIGNATURE----- --WSsksJ2MmDkORlvCVTjLliT0BeonjHg4D-- From owner-freebsd-scsi@freebsd.org Thu Dec 15 09:40:54 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B150CC80FB3 for ; Thu, 15 Dec 2016 09:40:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A12DF1D36 for ; Thu, 15 Dec 2016 09:40:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id uBF9esSs020126 for ; Thu, 15 Dec 2016 09:40:54 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-scsi@FreeBSD.org Subject: [Bug 211990] iscsi fails to reconnect and does not release devices Date: Thu, 15 Dec 2016 09:40:54 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: julien@perdition.city X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2016 09:40:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211990 --- Comment #34 from Julien Cigar --- No problems for more than a month now.. I tend to conclude that the MTU 9000 was the root cause of this problem (?) --=20 You are receiving this mail because: You are on the CC list for the bug.=