From owner-freebsd-net@freebsd.org Thu Aug 30 22:22:27 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50CB6EFA775 for ; Thu, 30 Aug 2018 22:22:27 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B1EF1751FF; Thu, 30 Aug 2018 22:22:26 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w7UMMIss028352; Thu, 30 Aug 2018 15:22:18 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w7UMMISG028351; Thu, 30 Aug 2018 15:22:18 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201808302222.w7UMMISG028351@pdx.rh.CN85.dnsmgr.net> Subject: Re: cxl nic not working after reboot In-Reply-To: To: Navdeep Parhar Date: Thu, 30 Aug 2018 15:22:18 -0700 (PDT) CC: Marius Halden , freebsd-net@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2018 22:22:27 -0000 > On 8/30/18 2:51 PM, Marius Halden wrote: > > On Thu, Aug 30, 2018, at 19:27, Navdeep Parhar wrote: > >> On 8/30/18 4:21 AM, Marius Halden wrote: > >>> I tried to downgrade to a the previous bsdrp version we were running based on 11.1-RELEASE-p10, but it did not start working again. ifconfig output and t5nex0 devlog follows (non-working state), but it all looks fine to me (media has always been reported wrong). I did notice though that "hw_mac_init_port[0], ?" will always be logged when it works but not when when it doesn't work. I confirmed this on a box still running the old version which functions as intended. > >>> > >>> The ones not working has a newer firmware version than the ones working. > >>> > >>> Non-working: > >>> # sysctl dev.t5nex.0.firmware_version > >>> dev.t5nex.0.firmware_version: 1.19.1.0 > >>> > >>> > >>> Working: > >>> # sysctl dev.t5nex.0.firmware_version > >>> dev.t5nex.0.firmware_version: 1.16.45.0 > >>> > >>> Any other ideas? > >>> > >> > >> I'm still looking into it. The current working theory is that the peer > >> is trying to autonegotiate when it shouldn't (it's 1Gbps optics). If > >> you know how to disable AN on the peer this should be easy to test. > > > > Thanks. Unfortunately we don't have access to the peer as that's our ISPs router. According to the sysctl autonegotiate is not supported on our side. > > > > I'll get a couple of 1G optics in a day or so and then I'll be able to > try some things for myself. Wait for that or try a 1G TWINAX (copper) > SFP+ cable in case you have one -- that should be able to link up (if > it's really autonegotiation that's causing problems). I have seen some issues if one end has 1G optics, and the other end has 1/10G optics, it wont reliably come up every time unless you nail the other end to 1Gbps. -- Rod Grimes rgrimes@freebsd.org