From owner-freebsd-drivers@FreeBSD.ORG Tue Jul 22 07:37:15 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBA5EA97; Tue, 22 Jul 2014 07:37:15 +0000 (UTC) Received: from mail-vc0-x230.google.com (mail-vc0-x230.google.com [IPv6:2607:f8b0:400c:c03::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 698D42669; Tue, 22 Jul 2014 07:37:15 +0000 (UTC) Received: by mail-vc0-f176.google.com with SMTP id id10so9404610vcb.35 for ; Tue, 22 Jul 2014 00:37:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=VUxw5qlgHyuHnhT9RRogIgzrono5Vp+RSMmbK1G18Wk=; b=luwFGJiYzEJl/MV8xMOEND5JmSWSjy5qaxrE8jmWH4ArmOR0irHhEGltFxupdV1YzJ fU/t6U3rBrhsh0yRy2uErrjjgpLbW+nKzKAVvFNf5NOiLsOP0cTIL8/HAZ4cQRkb/VIN /d0VYOdsdA722JGkFNnfvBWdCR1fhCOsAtQibeVWILEpFDZnXPKF7ee8GgKEjYPDDBwA Dkzh+bacXk6bk82BXHCZbjzFaIvCkDLzNhtwaduV+0AkTobnubSJaCUdDcbJLZUvEKMY +J81djgHOD276OXAOo5NMEGAZdpoOVCDddRQKjfT8YQyDEQWXTCv605bIjRWIBsMcD/2 Am3A== X-Received: by 10.52.184.199 with SMTP id ew7mr31515303vdc.44.1406014633905; Tue, 22 Jul 2014 00:37:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.156.71 with HTTP; Tue, 22 Jul 2014 00:36:53 -0700 (PDT) From: bharat singh Date: Tue, 22 Jul 2014 13:06:53 +0530 Message-ID: Subject: QLogic 24xx FC HBAs not working fine with CAM Target Layer To: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2014 07:37:15 -0000 Hello, I am trying to use a new firmware on a single port FC HBA with freebsd9 stable, but facing the firmware issue. Card came with firmware version 5.5.0 (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision 5.5.0) I upgraded the firmware to 5.6.5, and disabled firmware flag by setting "fwload_disable=YES" flag in /boot/device.hints to retain the newer firmware. On subsequent reboots the firmware fails to initialize and throws isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) [root@Ctrl4 ~]# ctladm port -l Port Online Type Name pp vp WWNN WWPN Speed Vendor 0 YES IOCTL CTL ioctl 0 0 0 0 0 1 YES INTERNAL ctl2cam 0 0 0x5000000edac0c700 0x5000000edac0c702 0 2 YES INTERNAL CTL internal 0 0 0 0 0 * It's not showing FC port 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 = 0x0008 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ isp_reset:1045) 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from ufs:/dev/label/root_0 [rw]... 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach returned 6 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci 3 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware isp_2400_multi from dmesg: isp0: RISC2HOST ISR 0x40000101 isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ isp_reset:1045) isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) isp0: RISC2HOST ISR 0x40000101 device_attach: isp0 attach returned 6 isp0: port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3 isp0: loaded firmware isp_2400_multi isp0: set PCI latency to 64 isp0: request/result area @ 0xf700000/0x38000 isp0: scdma @ 0xf6b9000/0x1000 isp0: IN mbox 0 = 0x0000 isp0: RISC2HOST ISR 0x0 isp0: RISC2HOST ISR 0x40008001 isp0: OUT mbox 0 = 0x4000 isp0: IN mbox 0 = 0x0006 isp0: IN mbox 1 = 0xdead The workaround is to enable the firmware flash by setting "fwload_disable=NO", but that doesn't serves my purpose. Is there a way to avoid the issue without a firmware flash on every reboot ? Moreover i have seen the issue on older firmware 5.5.0 also, as soon I disable the firmware flash and reboot. Some old threads, but not helping much. http://lists.freebsd.org/pipermail/freebsd-scsi/2010-April/004215.html Any help is deeply appreciated. -- Bharat Singh From owner-freebsd-drivers@FreeBSD.ORG Wed Jul 23 04:48:36 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A191DAE; Wed, 23 Jul 2014 04:48:36 +0000 (UTC) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3BF542D17; Wed, 23 Jul 2014 04:48:35 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.9/8.14.2) with ESMTP id s6N4MxiJ015145; Tue, 22 Jul 2014 22:22:59 -0600 (MDT) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.9/8.14.9/Submit) id s6N4Mw4m015144; Tue, 22 Jul 2014 22:22:58 -0600 (MDT) (envelope-from ken) Date: Tue, 22 Jul 2014 22:22:58 -0600 From: "Kenneth D. Merry" To: bharat singh Subject: Re: QLogic 24xx FC HBAs not working fine with CAM Target Layer Message-ID: <20140723042258.GA15089@nargothrond.kdm.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 04:48:36 -0000 On Tue, Jul 22, 2014 at 13:06:53 +0530, bharat singh wrote: > Hello, > > I am trying to use a new firmware on a single port FC HBA with freebsd9 > stable, but facing the firmware issue. > > Card came with firmware version 5.5.0 > (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision 5.5.0) > > I upgraded the firmware to 5.6.5, and disabled firmware flag by setting > "fwload_disable=YES" flag in /boot/device.hints to retain the newer > firmware. > On subsequent reboots the firmware fails to initialize and throws > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) In general, you want to load the ispfw module or compile it into your kernel. The driver was tested with the firmware included in the module, and may not work as well (or at all) with a different version. If you know what changed between the firmware version compiled into the ispfw module and the one you want to run, you can make any necessary driver modifications. > [root@Ctrl4 ~]# ctladm port -l > Port Online Type Name pp vp WWNN WWPN > Speed Vendor > 0 YES IOCTL CTL ioctl 0 0 0 0 > 0 > 1 YES INTERNAL ctl2cam 0 0 0x5000000edac0c700 > 0x5000000edac0c702 0 > 2 YES INTERNAL CTL internal 0 0 0 0 > 0 > > * It's not showing FC port You need to compile your kernel with: options ISP_TARGET_MODE=1 Otherwise you won't get target mode functionality in the isp(4) driver. > 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 = 0x0008 > 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times > 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x8) > Timeout (100000us) (started @ isp_reset:1045) > 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT FIRMWARE' > failed (TIMEOUT) > 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from > ufs:/dev/label/root_0 [rw]... > 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach returned 6 > 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: Adapter> port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at device 0.0 > on pci 3 > 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware isp_2400_multi > > from dmesg: > isp0: RISC2HOST ISR 0x40000101 > isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ > isp_reset:1045) > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > isp0: RISC2HOST ISR 0x40000101 > device_attach: isp0 attach returned 6 > isp0: port 0x3000-0x30ff mem > 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3 > isp0: loaded firmware isp_2400_multi > isp0: set PCI latency to 64 > isp0: request/result area @ 0xf700000/0x38000 > isp0: scdma @ 0xf6b9000/0x1000 > isp0: IN mbox 0 = 0x0000 > isp0: RISC2HOST ISR 0x0 > isp0: RISC2HOST ISR 0x40008001 > isp0: OUT mbox 0 = 0x4000 > isp0: IN mbox 0 = 0x0006 > isp0: IN mbox 1 = 0xdead > > The workaround is to enable the firmware flash by setting > "fwload_disable=NO", but that doesn't serves my purpose. > > Is there a way to avoid the issue without a firmware flash on every reboot ? > Moreover i have seen the issue on older firmware 5.5.0 also, as soon I > disable the firmware flash and reboot. If I recall correctly, the firmware that is loaded on boot is loaded into RAM on the card, not flashed. I don't think it affects what is actually flashed onto the board. So that really is what you want to do. Also, just FYI, in past testing with 4Gb Qlogic boards I was limited to 256MB/sec in each direction when I tried them out. I haven't seen the same limitation with the 8Gb boards. It has been a few years since I tried out target mode with a 4Gb board, but keep an eye out for that once you get it running. Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-drivers@FreeBSD.ORG Wed Jul 23 07:03:37 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C04B06B7; Wed, 23 Jul 2014 07:03:37 +0000 (UTC) Received: from mail-vc0-x231.google.com (mail-vc0-x231.google.com [IPv6:2607:f8b0:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 484A9286B; Wed, 23 Jul 2014 07:03:37 +0000 (UTC) Received: by mail-vc0-f177.google.com with SMTP id hy4so1360982vcb.8 for ; Wed, 23 Jul 2014 00:03:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=K5MFJFJ271vGIjJ02KnDAd9QEXBiwP0H6ZtHLj/714o=; b=0OtFRTnEU+qBgVbhTU3rs/ShO0nr7vgSSluLReuIfdedH7d4anlNH37t9FM74UdlFW OKtakWqYeKZwwUuzOSCZaD6rX8xueJrWo4FTh+3ZwBjDbP8Bb/ynnEfXdoOzP4vwFo0n 7oubSjzqwEsEtjpSgo0e3A8IHrzFpKjrEUh2hzjCk3/0LPov4Z+R1H8YpAsOobACN5eF GM+M9KQ9CE3i78ilzQKGQcqlPpIbeJwkhcZa6pzAWcdiw6B65ycHiq+51J8ya7i4sPnb 2qDdb0HCgBDTqSFk/UsbOPFUTxGpZFug4baPHqqqJlLpO8zA27n5Mc+Sc72izqQFmP4w Pp3A== X-Received: by 10.220.131.207 with SMTP id y15mr1089067vcs.71.1406099016297; Wed, 23 Jul 2014 00:03:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.156.71 with HTTP; Wed, 23 Jul 2014 00:03:16 -0700 (PDT) In-Reply-To: <20140723042258.GA15089@nargothrond.kdm.org> References: <20140723042258.GA15089@nargothrond.kdm.org> From: bharat singh Date: Wed, 23 Jul 2014 12:33:16 +0530 Message-ID: Subject: Re: QLogic 24xx FC HBAs not working fine with CAM Target Layer To: "Kenneth D. Merry" , mjacob@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 07:03:37 -0000 On Wed, Jul 23, 2014 at 9:52 AM, Kenneth D. Merry wrote: > On Tue, Jul 22, 2014 at 13:06:53 +0530, bharat singh wrote: > > Hello, > > > > I am trying to use a new firmware on a single port FC HBA with freebsd9 > > stable, but facing the firmware issue. > > > > Card came with firmware version 5.5.0 > > (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision 5.5.0) > > > > I upgraded the firmware to 5.6.5, and disabled firmware flag by setting > > "fwload_disable=YES" flag in /boot/device.hints to retain the newer > > firmware. > > On subsequent reboots the firmware fails to initialize and throws > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > > In general, you want to load the ispfw module or compile it into your > kernel. The driver was tested with the firmware included in the module, > and may not work as well (or at all) with a different version. > > If you know what changed between the firmware version compiled into the > ispfw module and the one you want to run, you can make any necessary driver > modifications. > > > [root@Ctrl4 ~]# ctladm port -l > > Port Online Type Name pp vp WWNN WWPN > > Speed Vendor > > 0 YES IOCTL CTL ioctl 0 0 0 0 > > 0 > > 1 YES INTERNAL ctl2cam 0 0 0x5000000edac0c700 > > 0x5000000edac0c702 0 > > 2 YES INTERNAL CTL internal 0 0 0 0 > > 0 > > > > * It's not showing FC port > > You need to compile your kernel with: > > options ISP_TARGET_MODE=1 > > Otherwise you won't get target mode functionality in the isp(4) driver. > Yes, ISP is enabled for target functionality. > > > 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 = 0x0008 > > 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > > 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times > > 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x8) > > Timeout (100000us) (started @ isp_reset:1045) > > 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT FIRMWARE' > > failed (TIMEOUT) > > 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > > 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from > > ufs:/dev/label/root_0 [rw]... > > 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach returned 6 > > 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: > Adapter> port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at device > 0.0 > > on pci 3 > > 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware isp_2400_multi > > > > from dmesg: > > isp0: RISC2HOST ISR 0x40000101 > > isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ > > isp_reset:1045) > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > > isp0: RISC2HOST ISR 0x40000101 > > device_attach: isp0 attach returned 6 > > isp0: port 0x3000-0x30ff mem > > 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3 > > isp0: loaded firmware isp_2400_multi > > isp0: set PCI latency to 64 > > isp0: request/result area @ 0xf700000/0x38000 > > isp0: scdma @ 0xf6b9000/0x1000 > > isp0: IN mbox 0 = 0x0000 > > isp0: RISC2HOST ISR 0x0 > > isp0: RISC2HOST ISR 0x40008001 > > isp0: OUT mbox 0 = 0x4000 > > isp0: IN mbox 0 = 0x0006 > > isp0: IN mbox 1 = 0xdead > > > > The workaround is to enable the firmware flash by setting > > "fwload_disable=NO", but that doesn't serves my purpose. > > > > Is there a way to avoid the issue without a firmware flash on every > reboot ? > > Moreover i have seen the issue on older firmware 5.5.0 also, as soon I > > disable the firmware flash and reboot. > > If I recall correctly, the firmware that is loaded on boot is loaded into > RAM on the card, not flashed. I don't think it affects what is actually > flashed onto the board. When I disable the firmware download by setting "fwload_disable=YES" and reboot, still most of the times it works. So i think its the eprom getting overwritten on every reboot. > So that really is what you want to do. > I disabled ISP, compiled the kernel with Qlogic driver and loaded its firmware. Since the Qlogic driver is not compatible with our CTL module, I need to go back to the old ISP driver. So I reverted my kernel and suppressed further firmware downloads by "fwload_disable=YES" and tried out. Having new firmware with old ISP driver is creating issues (mailbox timeouts) > > Also, just FYI, in past testing with 4Gb Qlogic boards I was limited to > 256MB/sec in each direction when I tried them out. I haven't seen the same > limitation with the 8Gb boards. > > It has been a few years since I tried out target mode with a 4Gb board, but > keep an eye out for that once you get it running. > > Ken > -- > Kenneth Merry > ken@FreeBSD.ORG > -- Bharat Singh Member Technical Staff, NetApp From owner-freebsd-drivers@FreeBSD.ORG Wed Jul 23 14:09:29 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7EE542ED; Wed, 23 Jul 2014 14:09:29 +0000 (UTC) Received: from mail-vc0-x235.google.com (mail-vc0-x235.google.com [IPv6:2607:f8b0:400c:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 05CB62ED0; Wed, 23 Jul 2014 14:09:28 +0000 (UTC) Received: by mail-vc0-f181.google.com with SMTP id lf12so2223294vcb.26 for ; Wed, 23 Jul 2014 07:09:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=hu39cMChUiv2LWFh/JDg8A9fvBkladnpvwqe4PERpAM=; b=c1sXSQUa0xltg8LVggIUklkWADEUj1MXoHCBxrkV0UuUXR7k5OmueMevyrV9lStpO4 DzJi9f7DjPlmXawKY1AKuHeOXEmhCbIb8pkgkD26t7P1XwLEjZTcqwGuHSrzHESic9uJ XscODlFvEaRsyK2MIbp5VjTcogsKaASbFw4NcPri4csNzVcJTa+O0cQfT9ucjv/PdX9c ksSFSfOjaw0ijDcsFp6uDuUDJBHjq47680/FoDXv7CrPuU/ALfMPKA3ydEVFlIZAJZD6 hl2vC4622X4zRscpH1hsbVi6HrjlHrhG6I79iSYdu/F31E2X54Xf9s4K8Gdh44XWF4u5 Zcvg== X-Received: by 10.52.154.106 with SMTP id vn10mr1978945vdb.36.1406124567928; Wed, 23 Jul 2014 07:09:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.156.71 with HTTP; Wed, 23 Jul 2014 07:09:07 -0700 (PDT) In-Reply-To: References: <20140723042258.GA15089@nargothrond.kdm.org> From: bharat singh Date: Wed, 23 Jul 2014 19:39:07 +0530 Message-ID: Subject: Re: QLogic 24xx FC HBAs not working fine with CAM Target Layer To: "Kenneth D. Merry" , Matthew Jacob Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-scsi@freebsd.org, freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 14:09:29 -0000 Some more info 8G HBA also showing same issues. Once firmware issue is resolved I am able to run I/O without any issue. Is it necessary to enable firmware download on every reboot. If I am suppressing the firmware download by setting the "fwload_disable=YES" it's causing the issue inconsistently. On Wed, Jul 23, 2014 at 12:33 PM, bharat singh wrote: > On Wed, Jul 23, 2014 at 9:52 AM, Kenneth D. Merry wrote: > >> On Tue, Jul 22, 2014 at 13:06:53 +0530, bharat singh wrote: >> > Hello, >> > >> > I am trying to use a new firmware on a single port FC HBA with freebsd9 >> > stable, but facing the firmware issue. >> > >> > Card came with firmware version 5.5.0 >> > (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision 5.5.0) >> > >> > I upgraded the firmware to 5.6.5, and disabled firmware flag by setting >> > "fwload_disable=YES" flag in /boot/device.hints to retain the newer >> > firmware. >> > On subsequent reboots the firmware fails to initialize and throws >> > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) >> >> In general, you want to load the ispfw module or compile it into your >> kernel. The driver was tested with the firmware included in the module, >> and may not work as well (or at all) with a different version. >> >> If you know what changed between the firmware version compiled into the >> ispfw module and the one you want to run, you can make any necessary >> driver >> modifications. >> >> > [root@Ctrl4 ~]# ctladm port -l >> > Port Online Type Name pp vp WWNN WWPN >> > Speed Vendor >> > 0 YES IOCTL CTL ioctl 0 0 0 0 >> > 0 >> > 1 YES INTERNAL ctl2cam 0 0 0x5000000edac0c700 >> > 0x5000000edac0c702 0 >> > 2 YES INTERNAL CTL internal 0 0 0 0 >> > 0 >> > >> > * It's not showing FC port >> >> You need to compile your kernel with: >> >> options ISP_TARGET_MODE=1 >> >> Otherwise you won't get target mode functionality in the isp(4) driver. >> > Yes, ISP is enabled for target functionality. > >> >> > 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 = 0x0008 >> > 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 >> > 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times >> > 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x8) >> > Timeout (100000us) (started @ isp_reset:1045) >> > 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT >> FIRMWARE' >> > failed (TIMEOUT) >> > 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 >> > 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from >> > ufs:/dev/label/root_0 [rw]... >> > 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach returned 6 >> > 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: > > Adapter> port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at device >> 0.0 >> > on pci 3 >> > 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware isp_2400_multi >> > >> > from dmesg: >> > isp0: RISC2HOST ISR 0x40000101 >> > isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ >> > isp_reset:1045) >> > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) >> > isp0: RISC2HOST ISR 0x40000101 >> > device_attach: isp0 attach returned 6 >> > isp0: port 0x3000-0x30ff mem >> > 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3 >> > isp0: loaded firmware isp_2400_multi >> > isp0: set PCI latency to 64 >> > isp0: request/result area @ 0xf700000/0x38000 >> > isp0: scdma @ 0xf6b9000/0x1000 >> > isp0: IN mbox 0 = 0x0000 >> > isp0: RISC2HOST ISR 0x0 >> > isp0: RISC2HOST ISR 0x40008001 >> > isp0: OUT mbox 0 = 0x4000 >> > isp0: IN mbox 0 = 0x0006 >> > isp0: IN mbox 1 = 0xdead >> > >> > The workaround is to enable the firmware flash by setting >> > "fwload_disable=NO", but that doesn't serves my purpose. >> > >> > Is there a way to avoid the issue without a firmware flash on every >> reboot ? >> > Moreover i have seen the issue on older firmware 5.5.0 also, as soon I >> > disable the firmware flash and reboot. >> >> If I recall correctly, the firmware that is loaded on boot is loaded into >> RAM on the card, not flashed. I don't think it affects what is actually >> flashed onto the board. > > When I disable the firmware download by setting "fwload_disable=YES" and > reboot, still most of the times it works. So i think its the eprom getting > overwritten on every reboot. > > >> > So that really is what you want to do. >> > > I disabled ISP, compiled the kernel with Qlogic driver and loaded its > firmware. Since the Qlogic driver is not compatible with our CTL module, I > need to go back to the old ISP driver. So I reverted my kernel and > suppressed further firmware downloads by "fwload_disable=YES" and tried > out. Having new firmware with old ISP driver is creating issues (mailbox > timeouts) > >> >> Also, just FYI, in past testing with 4Gb Qlogic boards I was limited to >> 256MB/sec in each direction when I tried them out. I haven't seen the >> same >> limitation with the 8Gb boards. >> >> It has been a few years since I tried out target mode with a 4Gb board, >> but >> keep an eye out for that once you get it running. >> >> Ken >> -- >> Kenneth Merry >> ken@FreeBSD.ORG >> > > > > -- > Bharat Singh > Member Technical Staff, NetApp > > > > -- Bharat Singh Member Technical Staff, NetApp From owner-freebsd-drivers@FreeBSD.ORG Wed Jul 23 15:00:17 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 961FBFD0 for ; Wed, 23 Jul 2014 15:00:17 +0000 (UTC) Received: from demig.de (demig.de [81.169.228.196]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "Parallels Panel", Issuer "Parallels Panel" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 19F8C2664 for ; Wed, 23 Jul 2014 15:00:16 +0000 (UTC) Received: (qmail 20259 invoked from network); 23 Jul 2014 16:52:32 +0200 Received: from ip-84-119-172-79.unity-media.net (HELO firewall.demig.intra) (84.119.172.79) by demig.de with ESMTPSA (DHE-RSA-AES256-SHA encrypted, authenticated); 23 Jul 2014 16:52:32 +0200 Received: from SRV-FS-2.Demig.intra (nameserver.demig.intra [192.168.148.248]) by firewall.demig.intra (8.14.4/8.14.4) with ESMTP id s6NEqUlx091514 for ; Wed, 23 Jul 2014 16:52:30 +0200 (CEST) (envelope-from nkoch@demig.de) Received: from [192.168.148.83] (192.168.148.83) by SRV-FS-2 (192.168.148.248) with Microsoft SMTP Server (TLS) id 14.3.123.3; Wed, 23 Jul 2014 16:52:25 +0200 Message-ID: <53CFCC28.5070302@demig.de> Date: Wed, 23 Jul 2014 16:52:24 +0200 From: Norbert Koch User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Subject: touch controller problem: uep_intr_callback not called X-Scanned-By: MIMEDefang 2.73 on 192.168.148.235 Content-Type: text/plain; charset="ISO-8859-15" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 15:00:17 -0000 Hello. I have here a USB touch controller which should be handled by the *uep *driver. This is what usbconfig shows: ugen0.2: at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON bLength = 0x0012 bDescriptorType = 0x0001 bcdUSB = 0x0110 bDeviceClass = 0x0000 bDeviceSubClass = 0x0000 bDeviceProtocol = 0x0000 bMaxPacketSize0 = 0x0040 idVendor = *0x0eef* idProduct = *0x0001* bcdDevice = *0x0100* iManufacturer = *0x0001 * iProduct = *0x0002 * iSerialNumber = 0x0000 bNumConfigurations = 0x0001 First scenario: *ums.ko loaded, uep.ko not loaded* When I connect the controller driver ums is attached (*/dev/ums0*). When I touch the screen I see incoming traffic with usbdump. The messages are not correctly recognized, which is to be expected. 16:40:22.902239 usbus0.2 DONE-INTR-EP=00000081,SPD=FULL,NFR=1,SLEN=8,IVAL=3,ERR=0 frame[0] READ 6 bytes 0000 01 *80 3A 0A F6 07* -- -- -- -- -- -- -- -- -- -- |..:... | 16:40:22.903240 usbus0.2 SUBM-INTR-EP=00000081,SPD=FULL,NFR=1,SLEN=0,IVAL=3 frame[0] READ 64 bytes 16:40:22.908189 usbus0.2 DONE-INTR-EP=00000081,SPD=FULL,NFR=1,SLEN=8,IVAL=3,ERR=0 frame[0] READ 6 bytes 0000 01 *81 3A 0A F8 07* -- -- -- -- -- -- -- -- -- -- |..:... | The byte sequences above seem to correspond to up/down touch events. Second scenario: *both ums.ko and uep.ko loaded* When I connect the controller driver uep is attached (*/dev/ums0*). Whe I now touch the screen I don't see any traffic coming in. After putting some printfs in uep.c, ums.c and usb_transfer.c I found that neither *uep_intr_callback*() nor usbpf_xfertap() are being called. As uep.c is quite similar to ums.c in probing/attaching I don't understand why this is so. Could any usb driver expert give me a hint where to search further? Thank you. Norbert Koch From owner-freebsd-drivers@FreeBSD.ORG Wed Jul 23 16:49:25 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5074C25; Wed, 23 Jul 2014 16:49:25 +0000 (UTC) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6823F2134; Wed, 23 Jul 2014 16:49:25 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.9/8.14.2) with ESMTP id s6NGnNpI020226; Wed, 23 Jul 2014 10:49:23 -0600 (MDT) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.9/8.14.9/Submit) id s6NGnNfp020225; Wed, 23 Jul 2014 10:49:23 -0600 (MDT) (envelope-from ken) Date: Wed, 23 Jul 2014 10:49:23 -0600 From: "Kenneth D. Merry" To: bharat singh Subject: Re: QLogic 24xx FC HBAs not working fine with CAM Target Layer Message-ID: <20140723164923.GA19987@nargothrond.kdm.org> References: <20140723042258.GA15089@nargothrond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-scsi@freebsd.org, mjacob@freebsd.org, davidcs@freebsd.org, freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 16:49:26 -0000 On Wed, Jul 23, 2014 at 12:33:16 +0530, bharat singh wrote: > On Wed, Jul 23, 2014 at 9:52 AM, Kenneth D. Merry wrote: > > > On Tue, Jul 22, 2014 at 13:06:53 +0530, bharat singh wrote: > > > Hello, > > > > > > I am trying to use a new firmware on a single port FC HBA with freebsd9 > > > stable, but facing the firmware issue. > > > > > > Card came with firmware version 5.5.0 > > > (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision 5.5.0) > > > > > > I upgraded the firmware to 5.6.5, and disabled firmware flag by setting > > > "fwload_disable=YES" flag in /boot/device.hints to retain the newer > > > firmware. > > > On subsequent reboots the firmware fails to initialize and throws > > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > > > > In general, you want to load the ispfw module or compile it into your > > kernel. The driver was tested with the firmware included in the module, > > and may not work as well (or at all) with a different version. > > > > If you know what changed between the firmware version compiled into the > > ispfw module and the one you want to run, you can make any necessary driver > > modifications. > > > > > [root@Ctrl4 ~]# ctladm port -l > > > Port Online Type Name pp vp WWNN WWPN > > > Speed Vendor > > > 0 YES IOCTL CTL ioctl 0 0 0 0 > > > 0 > > > 1 YES INTERNAL ctl2cam 0 0 0x5000000edac0c700 > > > 0x5000000edac0c702 0 > > > 2 YES INTERNAL CTL internal 0 0 0 0 > > > 0 > > > > > > * It's not showing FC port > > > > You need to compile your kernel with: > > > > options ISP_TARGET_MODE=1 > > > > Otherwise you won't get target mode functionality in the isp(4) driver. > > > Yes, ISP is enabled for target functionality. In order for CTL to recognize that an HBA is target-capable, the driver has to set the PIT_PROCESSOR bit (at least) in the target_sprt field in the XPT_PATH_INQ CCB. If you don't see the driver listed in the CTL port display, that means that the driver likely isn't reporting itself to be target-capable. See the code in isp_freebsd.c that fills in the XPT_PATH_INQ CCB. > > > 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 = 0x0008 > > > 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > > > 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times > > > 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x8) > > > Timeout (100000us) (started @ isp_reset:1045) > > > 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT FIRMWARE' > > > failed (TIMEOUT) > > > 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > > > 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from > > > ufs:/dev/label/root_0 [rw]... > > > 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach returned 6 > > > 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: > > Adapter> port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at device > > 0.0 > > > on pci 3 > > > 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware isp_2400_multi > > > > > > from dmesg: > > > isp0: RISC2HOST ISR 0x40000101 > > > isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ > > > isp_reset:1045) > > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > > > isp0: RISC2HOST ISR 0x40000101 > > > device_attach: isp0 attach returned 6 > > > isp0: port 0x3000-0x30ff mem > > > 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3 > > > isp0: loaded firmware isp_2400_multi > > > isp0: set PCI latency to 64 > > > isp0: request/result area @ 0xf700000/0x38000 > > > isp0: scdma @ 0xf6b9000/0x1000 > > > isp0: IN mbox 0 = 0x0000 > > > isp0: RISC2HOST ISR 0x0 > > > isp0: RISC2HOST ISR 0x40008001 > > > isp0: OUT mbox 0 = 0x4000 > > > isp0: IN mbox 0 = 0x0006 > > > isp0: IN mbox 1 = 0xdead > > > > > > The workaround is to enable the firmware flash by setting > > > "fwload_disable=NO", but that doesn't serves my purpose. > > > > > > Is there a way to avoid the issue without a firmware flash on every > > reboot ? > > > Moreover i have seen the issue on older firmware 5.5.0 also, as soon I > > > disable the firmware flash and reboot. > > > > If I recall correctly, the firmware that is loaded on boot is loaded into > > RAM on the card, not flashed. I don't think it affects what is actually > > flashed onto the board. > > When I disable the firmware download by setting "fwload_disable=YES" and > reboot, still most of the times it works. So i think its the eprom getting > overwritten on every reboot. Take a look at isp_reset() in isp.c. It downloads firmware into RAM on the card, and then tells the card to execute it. It isn't re-flashing the card. If that were the case, you would have to use a tool to flash new firmware to get back to the newer version of firmware. Instead, you just tell the driver not to download firmware and it uses the firmware that's flashed into the card. > > > So that really is what you want to do. > > > > I disabled ISP, compiled the kernel with Qlogic driver and loaded its > firmware. Since the Qlogic driver is not compatible with our CTL module, I > need to go back to the old ISP driver. So I reverted my kernel and > suppressed further firmware downloads by "fwload_disable=YES" and tried > out. Having new firmware with old ISP driver is creating issues (mailbox > timeouts) Which Qlogic driver are you using? If it isn't compatible with CTL, then it will likely take some work to add the ability to interface with CTL. > > > > Also, just FYI, in past testing with 4Gb Qlogic boards I was limited to > > 256MB/sec in each direction when I tried them out. I haven't seen the same > > limitation with the 8Gb boards. > > > > It has been a few years since I tried out target mode with a 4Gb board, but > > keep an eye out for that once you get it running. > > > > Ken > > -- > > Kenneth Merry > > ken@FreeBSD.ORG > > > > > > -- > Bharat Singh > Member Technical Staff, NetApp Ken -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-drivers@FreeBSD.ORG Wed Jul 23 16:51:49 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D87F6EAA; Wed, 23 Jul 2014 16:51:49 +0000 (UTC) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6895921D0; Wed, 23 Jul 2014 16:51:48 +0000 (UTC) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.14.9/8.14.2) with ESMTP id s6NGpmIY020272; Wed, 23 Jul 2014 10:51:48 -0600 (MDT) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.14.9/8.14.9/Submit) id s6NGpm4S020271; Wed, 23 Jul 2014 10:51:48 -0600 (MDT) (envelope-from ken) Date: Wed, 23 Jul 2014 10:51:48 -0600 From: "Kenneth D. Merry" To: bharat singh Subject: Re: QLogic 24xx FC HBAs not working fine with CAM Target Layer Message-ID: <20140723165147.GB19987@nargothrond.kdm.org> References: <20140723042258.GA15089@nargothrond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-scsi@freebsd.org, Matthew Jacob , davidcs@freebsd.org, freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 16:51:49 -0000 On Wed, Jul 23, 2014 at 19:39:07 +0530, bharat singh wrote: > Some more info > > 8G HBA also showing same issues. Once firmware issue is resolved I am able > to run I/O without any issue. > Is it necessary to enable firmware download on every reboot. Yes. The isp(4) driver at least is somewhat tied to a given firmware revision. If you update the firmware, you have to take into account the changes that occured in the intervening firmware releases. > If I am suppressing the firmware download by setting the > "fwload_disable=YES" it's causing the issue inconsistently. That isn't surprising. I would suggest either running with the firmware bundled with the driver, or update the driver to account for the changes in the firmware. (In that case, you'll want to bundle the new firmware with the driver, so you don't have to flash every card on every machine you have with the new firmware.) Ken > > On Wed, Jul 23, 2014 at 12:33 PM, bharat singh > wrote: > > > On Wed, Jul 23, 2014 at 9:52 AM, Kenneth D. Merry wrote: > > > >> On Tue, Jul 22, 2014 at 13:06:53 +0530, bharat singh wrote: > >> > Hello, > >> > > >> > I am trying to use a new firmware on a single port FC HBA with freebsd9 > >> > stable, but facing the firmware issue. > >> > > >> > Card came with firmware version 5.5.0 > >> > (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision 5.5.0) > >> > > >> > I upgraded the firmware to 5.6.5, and disabled firmware flag by setting > >> > "fwload_disable=YES" flag in /boot/device.hints to retain the newer > >> > firmware. > >> > On subsequent reboots the firmware fails to initialize and throws > >> > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > >> > >> In general, you want to load the ispfw module or compile it into your > >> kernel. The driver was tested with the firmware included in the module, > >> and may not work as well (or at all) with a different version. > >> > >> If you know what changed between the firmware version compiled into the > >> ispfw module and the one you want to run, you can make any necessary > >> driver > >> modifications. > >> > >> > [root@Ctrl4 ~]# ctladm port -l > >> > Port Online Type Name pp vp WWNN WWPN > >> > Speed Vendor > >> > 0 YES IOCTL CTL ioctl 0 0 0 0 > >> > 0 > >> > 1 YES INTERNAL ctl2cam 0 0 0x5000000edac0c700 > >> > 0x5000000edac0c702 0 > >> > 2 YES INTERNAL CTL internal 0 0 0 0 > >> > 0 > >> > > >> > * It's not showing FC port > >> > >> You need to compile your kernel with: > >> > >> options ISP_TARGET_MODE=1 > >> > >> Otherwise you won't get target mode functionality in the isp(4) driver. > >> > > Yes, ISP is enabled for target functionality. > > > >> > >> > 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 = 0x0008 > >> > 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > >> > 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times > >> > 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x8) > >> > Timeout (100000us) (started @ isp_reset:1045) > >> > 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT > >> FIRMWARE' > >> > failed (TIMEOUT) > >> > 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > >> > 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from > >> > ufs:/dev/label/root_0 [rw]... > >> > 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach returned 6 > >> > 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: >> > Adapter> port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at device > >> 0.0 > >> > on pci 3 > >> > 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware isp_2400_multi > >> > > >> > from dmesg: > >> > isp0: RISC2HOST ISR 0x40000101 > >> > isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ > >> > isp_reset:1045) > >> > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > >> > isp0: RISC2HOST ISR 0x40000101 > >> > device_attach: isp0 attach returned 6 > >> > isp0: port 0x3000-0x30ff mem > >> > 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3 > >> > isp0: loaded firmware isp_2400_multi > >> > isp0: set PCI latency to 64 > >> > isp0: request/result area @ 0xf700000/0x38000 > >> > isp0: scdma @ 0xf6b9000/0x1000 > >> > isp0: IN mbox 0 = 0x0000 > >> > isp0: RISC2HOST ISR 0x0 > >> > isp0: RISC2HOST ISR 0x40008001 > >> > isp0: OUT mbox 0 = 0x4000 > >> > isp0: IN mbox 0 = 0x0006 > >> > isp0: IN mbox 1 = 0xdead > >> > > >> > The workaround is to enable the firmware flash by setting > >> > "fwload_disable=NO", but that doesn't serves my purpose. > >> > > >> > Is there a way to avoid the issue without a firmware flash on every > >> reboot ? > >> > Moreover i have seen the issue on older firmware 5.5.0 also, as soon I > >> > disable the firmware flash and reboot. > >> > >> If I recall correctly, the firmware that is loaded on boot is loaded into > >> RAM on the card, not flashed. I don't think it affects what is actually > >> flashed onto the board. > > > > When I disable the firmware download by setting "fwload_disable=YES" and > > reboot, still most of the times it works. So i think its the eprom getting > > overwritten on every reboot. > > > > > >> > > So that really is what you want to do. > >> > > > > I disabled ISP, compiled the kernel with Qlogic driver and loaded its > > firmware. Since the Qlogic driver is not compatible with our CTL module, I > > need to go back to the old ISP driver. So I reverted my kernel and > > suppressed further firmware downloads by "fwload_disable=YES" and tried > > out. Having new firmware with old ISP driver is creating issues (mailbox > > timeouts) > > > >> > >> Also, just FYI, in past testing with 4Gb Qlogic boards I was limited to > >> 256MB/sec in each direction when I tried them out. I haven't seen the > >> same > >> limitation with the 8Gb boards. > >> > >> It has been a few years since I tried out target mode with a 4Gb board, > >> but > >> keep an eye out for that once you get it running. > >> > >> Ken > >> -- > >> Kenneth Merry > >> ken@FreeBSD.ORG > >> > > > > > > > > -- > > Bharat Singh > > Member Technical Staff, NetApp > > > > > > > > > > > -- > Bharat Singh > Member Technical Staff, NetApp -- Kenneth Merry ken@FreeBSD.ORG From owner-freebsd-drivers@FreeBSD.ORG Wed Jul 23 17:39:31 2014 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D4B34AC; Wed, 23 Jul 2014 17:39:31 +0000 (UTC) Received: from mail-vc0-x22e.google.com (mail-vc0-x22e.google.com [IPv6:2607:f8b0:400c:c03::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6C86226ED; Wed, 23 Jul 2014 17:39:30 +0000 (UTC) Received: by mail-vc0-f174.google.com with SMTP id la4so2783061vcb.5 for ; Wed, 23 Jul 2014 10:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=QPs7SMJceTuajMdJmjugg7p1QbiER72Ers3S0R8r5tE=; b=EopMe264P6euZZ68EOdd+umvNuuSL3Ds47NdcjmyYmAAihzD7jBOWEDlWIVe4gbRK1 ak87csvR2MMUAVydIVit901Fasd5KJDa37urG6BaQ1N6Oo8e0Ng7Bp2cBARJ1w6CK7A6 fO451iFh3aamz372Agbqj/Gx8upDYuD2gDcoVo1d9LvC7rQD9GcnZVDyXpdOp+5pg0H+ gSHXYSbqZCM8v+b3baKPsMDu7p4yD+t6iU+AMQhmMMMB+nxys9qntYYbrgAeU/itHxke A2XMP6pORmiDxcKYM60ptmZGp6B7NqQzjXeAinmDNST+plnT1m/c1hayf1qIHae+Wley lROQ== X-Received: by 10.221.69.8 with SMTP id ya8mr4273262vcb.39.1406137167601; Wed, 23 Jul 2014 10:39:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.58.156.71 with HTTP; Wed, 23 Jul 2014 10:39:07 -0700 (PDT) In-Reply-To: <20140723164923.GA19987@nargothrond.kdm.org> References: <20140723042258.GA15089@nargothrond.kdm.org> <20140723164923.GA19987@nargothrond.kdm.org> From: bharat singh Date: Wed, 23 Jul 2014 23:09:07 +0530 Message-ID: Subject: Re: QLogic 24xx FC HBAs not working fine with CAM Target Layer To: "Kenneth D. Merry" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18 Cc: freebsd-scsi@freebsd.org, Matthew Jacob , davidcs@freebsd.org, freebsd-drivers@freebsd.org X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 17:39:31 -0000 On Wed, Jul 23, 2014 at 10:19 PM, Kenneth D. Merry wrote: > On Wed, Jul 23, 2014 at 12:33:16 +0530, bharat singh wrote: > > On Wed, Jul 23, 2014 at 9:52 AM, Kenneth D. Merry > wrote: > > > > > On Tue, Jul 22, 2014 at 13:06:53 +0530, bharat singh wrote: > > > > Hello, > > > > > > > > I am trying to use a new firmware on a single port FC HBA with > freebsd9 > > > > stable, but facing the firmware issue. > > > > > > > > Card came with firmware version 5.5.0 > > > > (isp0: Board Type 2422, Chip Revision 0x3, resident F/W Revision > 5.5.0) > > > > > > > > I upgraded the firmware to 5.6.5, and disabled firmware flag by > setting > > > > "fwload_disable=3DYES" flag in /boot/device.hints to retain the new= er > > > > firmware. > > > > On subsequent reboots the firmware fails to initialize and throws > > > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > > > > > > In general, you want to load the ispfw module or compile it into your > > > kernel. The driver was tested with the firmware included in the > module, > > > and may not work as well (or at all) with a different version. > > > > > > If you know what changed between the firmware version compiled into t= he > > > ispfw module and the one you want to run, you can make any necessary > driver > > > modifications. > > > > > > > [root@Ctrl4 ~]# ctladm port -l > > > > Port Online Type Name pp vp WWNN WWPN > > > > Speed Vendor > > > > 0 YES IOCTL CTL ioctl 0 0 0 0 > > > > 0 > > > > 1 YES INTERNAL ctl2cam 0 0 0x5000000edac0c700 > > > > 0x5000000edac0c702 0 > > > > 2 YES INTERNAL CTL internal 0 0 0 0 > > > > 0 > > > > > > > > * It's not showing FC port > > > > > > You need to compile your kernel with: > > > > > > options ISP_TARGET_MODE=3D1 > > > > > > Otherwise you won't get target mode functionality in the isp(4) drive= r. > > > > > Yes, ISP is enabled for target functionality. > > In order for CTL to recognize that an HBA is target-capable, the driver h= as > to set the PIT_PROCESSOR bit (at least) in the target_sprt field in the > XPT_PATH_INQ CCB. > > If you don't see the driver listed in the CTL port display, that means th= at > the driver likely isn't reporting itself to be target-capable. > > See the code in isp_freebsd.c that fills in the XPT_PATH_INQ CCB. > > > > > 1448 Jul 22 11:05:37 Ctrl4 kernel: isp0: IN mbox 0 =3D 0x0008 > > > > 1450 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > > > > 1451 Jul 22 11:05:37 Ctrl4 last message repeated 999 times > > > > 1454 Jul 22 11:05:37 Ctrl4 kernel: isp0: Polled Mailbox Command (0x= 8) > > > > Timeout (100000us) (started @ isp_reset:1045) > > > > 1456 Jul 22 11:05:37 Ctrl4 kernel: isp0: Mailbox Command 'ABOUT > FIRMWARE' > > > > failed (TIMEOUT) > > > > 1458 Jul 22 11:05:37 Ctrl4 kernel: isp0: RISC2HOST ISR 0x40000101 > > > > 1460 Jul 22 11:05:37 Ctrl4 kernel: Trying to mount root from > > > > ufs:/dev/label/root_0 [rw]... > > > > 1462 Jul 22 11:05:37 Ctrl4 kernel: device_attach: isp0 attach > returned 6 > > > > 1464 Jul 22 11:05:37 Ctrl4 kernel: isp0: > > > Adapter> port 0x3000-0x30ff mem 0xc1040000-0xc1043fff irq 16 at > device > > > 0.0 > > > > on pci 3 > > > > 1465 Jul 22 11:05:37 Ctrl4 kernel: isp0: loaded firmware > isp_2400_multi > > > > > > > > from dmesg: > > > > isp0: RISC2HOST ISR 0x40000101 > > > > isp0: Polled Mailbox Command (0x8) Timeout (100000us) (started @ > > > > isp_reset:1045) > > > > isp0: Mailbox Command 'ABOUT FIRMWARE' failed (TIMEOUT) > > > > isp0: RISC2HOST ISR 0x40000101 > > > > device_attach: isp0 attach returned 6 > > > > isp0: port 0x3000-0x30ff mem > > > > 0xc1040000-0xc1043fff irq 16 at device 0.0 on pci3 > > > > isp0: loaded firmware isp_2400_multi > > > > isp0: set PCI latency to 64 > > > > isp0: request/result area @ 0xf700000/0x38000 > > > > isp0: scdma @ 0xf6b9000/0x1000 > > > > isp0: IN mbox 0 =3D 0x0000 > > > > isp0: RISC2HOST ISR 0x0 > > > > isp0: RISC2HOST ISR 0x40008001 > > > > isp0: OUT mbox 0 =3D 0x4000 > > > > isp0: IN mbox 0 =3D 0x0006 > > > > isp0: IN mbox 1 =3D 0xdead > > > > > > > > The workaround is to enable the firmware flash by setting > > > > "fwload_disable=3DNO", but that doesn't serves my purpose. > > > > > > > > Is there a way to avoid the issue without a firmware flash on every > > > reboot ? > > > > Moreover i have seen the issue on older firmware 5.5.0 also, as soo= n > I > > > > disable the firmware flash and reboot. > > > > > > If I recall correctly, the firmware that is loaded on boot is loaded > into > > > RAM on the card, not flashed. I don't think it affects what is > actually > > > flashed onto the board. > > > > When I disable the firmware download by setting "fwload_disable=3DYES" = and > > reboot, still most of the times it works. So i think its the eprom > getting > > overwritten on every reboot. > > Take a look at isp_reset() in isp.c. It downloads firmware into RAM on t= he > card, and then tells the card to execute it. It isn't re-flashing the > card. > > If that were the case, you would have to use a tool to flash new firmware > to get back to the newer version of firmware. Instead, you just tell the > driver not to download firmware and it uses the firmware that's flashed > into the card. > > Yes isp_reset() skips firmware download if firmware is not there or its overridden by the flag. In my setup I haven't flashed the firmware. After I load the newer firmware 5.6.5 and suppress further downloads, the new firmware exists even after reboots. Does it means that my HBA card came with 5.6.5 or the loaded firmware get persistent somewhere. > > > > So that really is what you want to do. > > > > > > > I disabled ISP, compiled the kernel with Qlogic driver and loaded its > > firmware. Since the Qlogic driver is not compatible with our CTL module= , > I > > need to go back to the old ISP driver. So I reverted my kernel and > > suppressed further firmware downloads by "fwload_disable=3DYES" and tri= ed > > out. Having new firmware with old ISP driver is creating issues (mailbo= x > > timeouts) > > Which Qlogic driver are you using? > I am trying to use a Qlogic version 5.6.5, but it doesn't integrate with the CTL of freebsd. It has all the functionality to manage the Qlogic card =E2=80=9Cmessages:May 26 17:52:33 es175 kernel: isp0: Board Type 2532, Chip Revision 0x2, resident F/W Revision 5.6.5=E2=80=9D > If it isn't compatible with CTL, then it will likely take some work to ad= d > the ability to interface with CTL. > > > > > > > Also, just FYI, in past testing with 4Gb Qlogic boards I was limited = to > > > 256MB/sec in each direction when I tried them out. I haven't seen th= e > same > > > limitation with the 8Gb boards. > > > > > > It has been a few years since I tried out target mode with a 4Gb > board, but > > > keep an eye out for that once you get it running. > > > > > > Ken > > > -- > > > Kenneth Merry > > > ken@FreeBSD.ORG > > > > > > > > > > > -- > > Bharat Singh > > Member Technical Staff, NetApp > > Ken > -- > Kenneth Merry > ken@FreeBSD.ORG > --=20 Bharat Singh Member Technical Staff, NetApp