From nobody Mon Feb 28 20:03:39 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4B60019E45E5 for ; Mon, 28 Feb 2022 20:03:41 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost1.sentex.ca", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4K6rt44JNPz4hcl; Mon, 28 Feb 2022 20:03:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [199.212.134.19]) by smarthost1.sentex.ca (8.16.1/8.16.1) with ESMTPS id 21SK3d6e066312 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Mon, 28 Feb 2022 15:03:40 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPV6:2607:f3e0:0:4::29] ([IPv6:2607:f3e0:0:4:0:0:0:29]) by pyroxene2a.sentex.ca (8.16.1/8.15.2) with ESMTPS id 21SK3dEa046473 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Mon, 28 Feb 2022 15:03:39 -0500 (EST) (envelope-from mike@sentex.net) Message-ID: <3c4c4cef-e737-5b09-92b3-d7d74d8690bb@sentex.net> Date: Mon, 28 Feb 2022 15:03:39 -0500 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.6.1 Subject: Re: cxgbe firmware error on latest RELENG_13 Content-Language: en-US To: Navdeep Parhar , FreeBSD-STABLE Mailing List References: <49736514-6500-4c7d-889d-15697d934f04@sentex.net> <8f3a6c6a-8d4c-7802-49a8-f0b8a6635723@FreeBSD.org> <0c37c656-3eeb-006f-118b-e06067affe81@sentex.net> From: mike tancsa In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.84 X-Rspamd-Queue-Id: 4K6rt44JNPz4hcl X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:1::12 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-3.40 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sentex.net]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-stable]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[199.212.134.19:received] X-ThisMailContainsUnwantedMimeParts: N On 2/28/2022 2:51 PM, Navdeep Parhar wrote: > On 2/28/22 11:48 AM, mike tancsa wrote: >> On 2/28/2022 2:43 PM, Navdeep Parhar wrote: >>> On 2/28/22 11:36 AM, mike tancsa wrote: >>>> Hi Navdeep, >>>> >>>>      I updated a pair of releng13 boxes today as I saw there are >>>> new drivers/firmware. However, the firmware seems to fail on both >>>> in different ways >>>> >>>> t5nex0: mem >>>> 0x91300000-0x9137ffff,0x90000000-0x90ffffff,0x91984000-0x91985fff >>>> irq 17 at device 0.4 on pci2 >>>> t5nex0: failed to load firmware module: 2. cfg 0, fw 0 >>>> t5nex0: firmware on card (1.25.0.40) is older than the version >>>> bundled with this driver, but the driver cannot take corrective >>>> action because it is unable to load the firmware module. >>> >>> It looks like the firmware KLD isn't available even though you have >>> t5fw_cfg_load="yes" in loader.conf. >>> >>> Can you try removing the cxgbe lines from loader.conf temporarily >>> and kldload if_cxgbe manually after boot?  It should be able to find >>> the KLD with the firmware and update the card.  Then restore the >>> lines in loader.conf and see if it comes up properly. >>> >> Hi, >> >>      It worked on the 4 port but not the 2 port unit >> >> root@coldstorage1:~ # kldload if_cxgbe >> t5nex0: mem >> 0xb2300000-0xb237ffff,0xb1000000-0xb1ffffff,0xb28a0000-0xb28a1fff irq >> 16 at device 0.4 on pci1 >> t5nex0: failed to connect to the firmware: 6, 0.  PCIE_FW 0x80000000 >> t5nex0: error during attach, adapter is now in recovery mode. >> root@coldstorage1:~ # > > Try a manual firmware upgrade while it's in recovery mode.  The > firmwares are the sys/dev/cxgbe/firmware/*.bin files in the src tree. > > # cxgbetool t5nex0 loadfw .../t5fw-1.26.6.0.bin > > Then reboot the system. > it seems to load it ok, but no luck after a soft reboot root@coldstorage1:/usr/src/sys/dev/cxgbe/firmware # cxgbetool t5nex0 loadfw t5fw-1.26.6.0.bin root@coldstorage1:/usr/src/sys/dev/cxgbe/firmware # md5 t5fw-1.26.6.0.bin MD5 (t5fw-1.26.6.0.bin) = 2a11a3d808b37d31e9caf19f75976e5b root@coldstorage1:/usr/src/sys/dev/cxgbe/firmware # But I did a full power cycle and it seemed to work after that! root@coldstorage1:~ # kldload if_cxgbe t5nex0: mem 0xb2300000-0xb237ffff,0xb1000000-0xb1ffffff,0xb28a0000-0xb28a1fff irq 16 at device 0.4 on pci1 t5nex0: PCIe gen3 x8, 2 ports, 14 MSI-X interrupts, 30 eq, 13 iq cxl0: on t5nex0 cxl0: 4 txq, 4 rxq (NIC); 4 txq (TOE), 2 rxq (TOE) cxl1: on t5nex0 cxl1: 4 txq, 4 rxq (NIC); 4 txq (TOE), 2 rxq (TOE) root@coldstorage1:~ # Thank you very much for your quick help to get me back up and running     ---Mike